Thinking Forward: Memory-Efficient Federated Finetuning of Language Models
Published @ NeurIPS, 2024September 2024Spry is a federated learning algorithm that enables finetuning LLMs using Forward-mode Auto Differentiation; to achieve low memory footprint, high accuracy, and fast convergence.