Heavy Tails in Correlated Sums
Correlation lets rare large steps align across many terms, and this alignment makes extreme outcomes occur far more often than independence predicts.
16 min 8 scenes
Explore by topic
Math & Logic
Mathematics, probability, statistics, formal logic, and algorithmic principles.
Why the Bell Curve Disappears
The bell curve appears only when the Central Limit Theorem can act: it requires enough independent samples, and a source whose variance is finite and not dominated by rare extreme events.
14 min 7 scenes
Escaping Saddle Points: Noise in Optimization
The mechanism by which random noise added to gradients or parameters systematically breaks the symmetry of a saddle point and drives the iterate toward a descending direction.
18 min 9 scenes
Lab Notebook: Why Sample Means Go Bell-Shaped
Sample means concentrate around the true mean with spread σ/√n because summing independent values convolves their distributions, an operation that erases sharp edges and corners until only a Gaussian profile remains.
18 min 9 scenes
평균이 왜 종 모양을 만드는가
종 모양은 숫자들 자체에서 오는 것이 아니라, 독립적인 흔들림들을 함께 더하는 평균 행위에서 비롯됩니다.
8 min 4 scenes
Where Does 1/√n Shrinkage Show Up?
The standard error of a mean shrinks as 1/√n, and this same scaling shows up visibly in dice averages, polling margins, and physics timing experiments.
18 min 9 scenes
Where Bell Curves Hide in Plain Sight
Bell-shaped distributions appear whenever a single result is the sum of many small, independent influences — and the Central Limit Theorem explains why this happens across heights, test scores, measurement errors, and beyond.
16 min 8 scenes
Why a Small Gradient Can Stall at a Saddle
Explains why gradient magnitude is the wrong progress signal near saddles: the gradient can vanish at a saddle while a negative-curvature direction still permits escape, and detecting that direction — not just measuring gradient size — is what drives rapid exit.
18 min 9 scenes
When Opportunity Cost Breaks Down
Opportunity cost delivers sharp answers only inside a narrow boundary of comparable, reversible, scale-compatible choices — and it breaks down precisely when decisions become identity-laden, irreversible, or incommensurable.
14 min 7 scenes
When Opportunity Cost Fails
Opportunity cost reliably predicts choice only when alternatives are fully known, commensurable, and psychologically available; outside those conditions, decision-makers substitute reference points, sunk anchors, or identity-based reasoning instead.
16 min 8 scenes
Saddle Points in the Wild
A saddle point is a stationary point with both upward and downward curvature, and its escape routes lie along the negative-curvature directions revealed by the Hessian's eigenvalues.
14 min 7 scenes
When the Central Limit Theorem Wobbles
The bell-shaped limit survives mild dependence but changes shape — width, tail weight, and even symmetry — as correlation length and strength grow, and a single correlation time scale controls the transition.
16 min 8 scenes
Saddle Points and Eigenvalue Escape Routes
At a saddle, the Hessian is a symmetric matrix whose eigenvalues are the second-derivative curvatures along its eigenvectors; positive eigenvalues mean stable directions and negative eigenvalues mean escape directions, with eigenvectors giving the exact axes of motion.
14 min 7 scenes
What Really Moves the Weights?
Effective step size is the product of learning rate and gradient magnitude, so step length scales with both knob and slope.
18 min 9 scenes
Does the Central Limit Theorem Survive Skew and Heavy Tails?
The CLT holds for the sample mean under any finite-variance source distribution, but convergence to normality is slow for skewed and especially heavy-tailed data — so 'large enough n' depends on the tail, not just the sample size.
18 min 9 scenes
What You Give Up vs. What You Pay
Opportunity cost is the value of the next-best alternative you forgo, while price is the money exchanged; the two diverge whenever time, attention, or forgone options carry value beyond the transaction itself.
16 min 8 scenes
Why Averages Always Make a Bell
The bell shape comes from adding independent wiggles together, not from the numbers themselves being bell-shaped.
16 min 8 scenes
Why Sample Means Look Normal
Averaging independent values smooths randomness: sample means concentrate around the true mean with spread shrinking as 1/√n, so the sampling distribution becomes approximately Gaussian regardless of the source's shape.
18 min 9 scenes
Why Gradient Descent Slows at Saddles
Gradient descent slows near a saddle because one direction has weak downhill slope and weak curvature, so the update becomes very small in that direction.
8 min 4 scenes
Learning Rate or Bad Model? Diagnosing Training Failure
A trained eye reads loss curves, gradient norms, and weight updates to separate learning-rate failure from architectural or data failure — and knows which knob to turn next.
14 min 7 scenes
When a Learning Rate Diverges
A learning rate that worked can diverge when the loss landscape shifts under it — when gradients spike, curvature grows, or accumulated updates push the model into a steeper region — and you can read early warning signs before the explosion.
24 min 12 scenes
Why Gradient Descent Slows Near Saddle Points
A saddle point stalls gradient descent because the gradient itself nearly vanishes, and the surrounding curvature decides whether the algorithm escapes or gets stuck.
20 min 10 scenes
Why Stochastic Gradient Descent Beats Full-Batch
Gradient noise in SGD is not a bug to fix but a feature to exploit: it provides implicit regularization that helps the optimizer escape saddle points, traverse flat regions, and prefer wide minima that generalize.
18 min 9 scenes
Signal or Noise? How Statistics Tests Whether Patterns Are Real
Statistical hypothesis testing provides a framework for distinguishing real patterns from chance ones by measuring how often data with no real pattern would look just as interesting.
16 min 8 scenes