Numerical Sensitivity: Why FP16 Breaks NAdam and How to Fix It
A visual deep dive into floating-point arithmetic, numerical stability, and why certain optimizers fail catastrophically in half precision. Learn the fundamentals of machine epsilon, catastrophic cancellation, and condition numbers through interactive visualizations.
