arXiv 2024
Liger SwiGLU
Liger Kernel: Fused SwiGLU
Pin-Lun Hsu, Yun Dai, +8
Trace Liger Kernel’s fused SiLU and multiply, derive both SwiGLU gradients, and separate saved activation memory from measured GPU performance.
Explore machine learning papers and reviews related to Memory. Find insights, analysis, and implementation details.
Liger SwiGLU
Pin-Lun Hsu, Yun Dai, +8
Trace Liger Kernel’s fused SiLU and multiply, derive both SwiGLU gradients, and separate saved activation memory from measured GPU performance.
Horace He
Deep learning performance optimization from first principles. Learn to identify compute-bound, memory-bound, and overhead bottlenecks with fusion techniques.