Song Han
MIT associate professor and NVIDIA research director; co-author of AWQ, SmoothQuant and StreamingLLM
- Associate Professor with tenure, EECS, Massachusetts Institute of Technology12
- Research Director (leads the Efficient AI team), NVIDIA3
Song Han is an associate professor with tenure in MIT's Department of Electrical Engineering and Computer Science and a research director at NVIDIA, where Han leads the Efficient AI team. Han's work centers on efficient AI computing, from the Deep Compression pruning and quantization techniques and the Efficient Inference Engine to TinyML methods for fitting neural networks on microcontrollers. More recently, Han co-authored AWQ, a 4-bit weight quantization method whose MIT-licensed repository includes TinyChat for on-device inference on edge GPUs, along with SmoothQuant and StreamingLLM for more efficient language-model inference. Han also developed the open lecture series EfficientML.ai, home of MIT's TinyML and Efficient AI Computing course, and has received the NSF CAREER Award and a Sloan Research Fellowship.123568104
Work
AWQ (Activation-aware Weight Quantization) and TinyChatOpen-source code
Co-author of the AWQ paper (2023) with Ji Lin, Jiaming Tang and others, which also presents TinyChat, an inference framework for 4-bit on-device language and vision-language models. The MIT-licensed llm-awq repository describes TinyChat as a turn-key solution for on-device inference on resource-constrained edge platforms and shows it running on NVIDIA Jetson Orin, and Han's MIT page notes the paper's Best Paper Award at MLSys 2024.5671
SmoothQuant (external site: github.com)Open-source code
Co-author of SmoothQuant (2022), a training-free 8-bit post-training quantization method for large language models; the code is published by MIT HAN Lab under the MIT license.891
StreamingLLM (external site: github.com)Open-source code
Co-author with Guangxuan Xiao, Yuandong Tian, Beidi Chen and Mike Lewis of "Efficient Streaming Language Models with Attention Sinks" (2023), which keeps attention-sink tokens in the KV cache so a model can generate long text with a fixed memory budget; the code is MIT-licensed.10111
TinyML and Efficient AI Computing (MIT 6.5940, EfficientML.ai) (external site: efficientml.ai)Published research
Developed the open lecture series EfficientML.ai, the course site for MIT 6.5940 TinyML and Efficient AI Computing, to share advances in efficient machine learning research.14
This profile covers public professional work only, from the sources below. It is not an endorsement or a ranking. To correct or remove it, use the correction form.