Explore machine learning papers and reviews related to Large Language Models. Find insights, analysis, and implementation details.
Mohammad Shoeybi, Mostofa Patwary, +4
Megatron-LM splits each transformer layer across GPUs with two all-reduces forward and two backward, and trains an 8.3B-parameter GPT-2 on 512 V100s.
Min Si, Pavan Balaji, +37
NCCLX is Meta's communication stack for Llama 4: host-driven, zero-copy collectives that start a 96K-GPU job 11x faster and cut decode time by up to 83%.
Peng Wang, Shuai Bai, +17
How Qwen2-VL perceives images and video at any resolution with naive dynamic resolution (variable visual tokens) and M-RoPE, a multimodal rotary position embedding that decomposes position into temporal, height, and width components.
Haotian Liu, Chunyuan Li, +2
LLaVA paper: align LLMs with visual information through instruction tuning on image-text pairs, enabling multimodal understanding and reasoning.