Skip to main content

Large Language Models

Explore machine learning papers and reviews related to Large Language Models. Find insights, analysis, and implementation details.

  • Tagged with
  • 4 papers
Back to all papers

Papers Related to Large Language Models

arXiv 2019

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

Mohammad Shoeybi, Mostofa Patwary, +4

Megatron-LM splits each transformer layer across GPUs with two all-reduces forward and two backward, and trains an 8.3B-parameter GPT-2 on 512 V100s.

arXiv 2025

Collective Communication for 100k+ GPUs

Min Si, Pavan Balaji, +37

NCCLX is Meta's communication stack for Llama 4: host-driven, zero-copy collectives that start a 96K-GPU job 11x faster and cut decode time by up to 83%.

arXiv 2024

Qwen2-VL: Vision-Language Perception at Any Resolution

Peng Wang, Shuai Bai, +17

How Qwen2-VL perceives images and video at any resolution with naive dynamic resolution (variable visual tokens) and M-RoPE, a multimodal rotary position embedding that decomposes position into temporal, height, and width components.

Mastodon