Skip to content

Source paper

LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models

Yaowei Zheng, Richong Zhang, Junhao Zhang and 2 more

Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo

arXiv 2403.13372 · 2024175 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

This paper introduces LLaMA-Factory, a unified framework designed to democratize and streamline the efficient fine-tuning of over 100 large language models (LLMs). By modularizing model loading, data processing, and training, and integrating a web-based UI (LLaMA Board), the authors enable users to apply state-of-the-art techniques like LoRA, GaLore, and QLoRA with minimal coding effort.

How it works (AI summary)

The framework is built on three decoupled modules: a 'Model Loader' that handles initialization and adapter attachment, a 'Data Worker' that standardizes diverse datasets via a pipeline, and a 'Trainer' that integrates efficient optimization methods (e.g., LoRA, BAdam) and training objectives (SFT, DPO, PPO). It utilizes a 'Model-Sharing RLHF' approach to enable reinforcement learning on consumer devices by dynamically switching adapters.