SustainFed-LLM: Renewable Energy Aware Client Selection for Energy Efficient Federated Training of Large Language Models

Federated Learning (FL) offers a decentralized approach to training large language models (LLMs), addressing critical concerns around data privacy and transmission costs. However, FL's inherent distributed training paradigm can lead to increased energy consumption and carbon emissions, especially with random or uninformed client selection. To tackle this issue, we propose SustainFed-LLM, a novel Q-learning-based client selection framework that integrates real-time renewable energy availability, carbon intensity data, and fairness considerations. By dynamically assessing client performance, spare capacity, and sustainability metrics, SustainFed-LLM optimizes client participation to minimize environmental impact while maintaining model accuracy. SustainFed-LLM significantly reduces energy consumption by up to 50%, while achieving convergence 30-70% faster compared to conventional selection strategies. We also analyze the communication overhead and computation cost, finding 30% fewer transmitted bytes and a 14% drop in FLOPS. The proposed framework also promotes fairer client participation, as evidenced by a reduced Gini coefficient. These findings underscore the potential of SustainFed-LLM to advance green AI, providing an effective pathway for large-scale sustainable and energy-efficient LLM training.