Cho-Jui Hsieh
UCLA associate professor of computer science; co-author of the LAMB and Lion optimizer papers
- Associate Professor of Computer Science, University of California, Los Angeles1
Cho-Jui Hsieh is an associate professor of computer science at UCLA; Hsieh's homepage says Hsieh was previously an assistant professor of computer science and statistics at UC Davis for three years and lists a visiting scholar role at Google beginning in summer 2018. Hsieh develops algorithms and optimization techniques for large-scale machine learning, including work on model size, training and prediction speed, and robustness of deep learning models. Hsieh is a co-author of Large Batch Optimization for Deep Learning: Training BERT in 76 minutes, which introduces the LAMB optimizer, and of Symbolic Discovery of Optimization Algorithms, which describes the Lion optimizer found through program search. With Pin-Yu Chen, Hsieh co-authored the book Adversarial Robustness for Machine Learning.123
Work
Lion optimizer (Symbolic Discovery of Optimization Algorithms) (external site: arxiv.org)Published research
Co-author of the 2023 paper that formulates algorithm discovery as program search and reports the Lion (EvoLved Sign Momentum) optimizer, which tracks only momentum and is more memory-efficient than Adam.3
LAMB optimizer (Large Batch Optimization for Deep Learning) (external site: arxiv.org)Published research
Co-author of the 2019 paper that develops LAMB, a layerwise adaptive large batch optimization technique, and provides convergence analysis for LAMB and LARS.2
Earlier roles
This profile covers public professional work only, from the sources below. It is not an endorsement or a ranking. To correct or remove it, use the correction form.