ICLR 2022
LoRA: Low-Rank Adaptation of Large Language Models
Edward J. Hu, Yelong Shen, +6
How LoRA adapts a frozen large language model by learning a low-rank update ΔW = (α/r)·BA to each weight matrix, training under 1% of the parameters of full fine-tuning and adding zero inference latency once merged.
- Parameter Efficiency
- Transformers
- Fine-Tuning
- +2 more tags