Jeremy Howard
Co-founder of Answer.AI, which released FSDP/QLoRA for fine-tuning large models on gaming GPUs
Jeremy Howard co-founded Answer.AI, an AI research and development lab, is a founding researcher at fast.ai, and created the fastai deep learning library. Answer.AI's first project, FSDP/QLoRA, is an open-source system for fine-tuning a 70-billion-parameter language model on a desktop computer with two 24 GB gaming GPUs. Answer.AI's announcement describes Howard discussing the combination of FSDP and QLoRA with Tim Dettmers, working with Hugging Face's Titus von Koeller to document the problems that arose, and writing a minimal standalone fine-tuning script for the project.123
Work
FSDP/QLoRA (external site: github.com)
Answer.AI's announcement credits Howard with discussing the idea of combining FSDP and QLoRA in late 2023 and with exploring and documenting the issues that occurred when the two were combined. The Apache-2.0 repository documents fine-tuning Llama 2 70B on two 24 GB GPUs, with 4-bit quantization through bitsandbytes or HQQ.345
Hugging Face Transformers in FSDP/QLoRA
According to the announcement, Howard studied how Meta's Llama-Recipes worked with bitsandbytes and Hugging Face's PEFT, Transformers, and Accelerate libraries, then built a minimal standalone script covering every step needed to fine-tune a model. The project's installation steps use the Transformers library.34
This profile covers public professional work only, from the sources below. It is not an endorsement or a ranking. To correct or remove it, use the correction form.