Personalized Collaborative Fine-Tuning for On-Device Large Language Models

Nicolas Wagner, Dongyang Fan, Martin Jaggi

Conference On Language Models 1 pre-proceedings (COLM 2024) Conference

BibTeX download is not available in the pre-proceeding

We explore on-device collaborative fine-tuning of large language models under limited local data availability. We introduce three distinct dynamic collaborator selection schemes, allowing trust-weighted personalized update aggregation: model-similarity-based, prediction-similarity-based and validation-performance-based. To minimize communication overhead, we integrate Low-Rank Adaptation (LoRA) and only exchange LoRA model updates. Our protocols, driven by prediction and performance metrics, surpass both FedAvg and local fine-tuning methods, which is particularly evident in realistic distributed scenarios with more diverse local data distributions. The results underscore the effectiveness of our approach in addressing heterogeneity and scarcity of the local datasets.