Azalia Mirhoseini
Stanford assistant professor directing Scaling Intelligence; co-founder of Ricursive Intelligence
- Assistant Professor of Computer Science; directs the Scaling Intelligence lab, Stanford University12
- Co-founder, Ricursive Intelligence12
Azalia Mirhoseini is an Assistant Professor of Computer Science at Stanford University, where Mirhoseini directs Scaling Intelligence, a lab working on scalable and self-improving AI systems, and is a co-founder of Ricursive Intelligence, a lab using AI to design chips. Mirhoseini previously spent several years at Google Brain, Anthropic and Google DeepMind, working on projects including Gemini and Claude. Mirhoseini's published work includes the sparsely-gated Mixture-of-Experts layer, deep reinforcement learning for chip placement (the AlphaChip line of work), and research on scaling inference compute through repeated sampling.12345
Research
Mirhoseini describes a research interest in self-improving AI, including AI that helps design models, software and hardware. Early work introduced the sparsely-gated Mixture-of-Experts layer for language models in 2017, alongside deep reinforcement learning methods for placing model computations across devices. Mirhoseini co-founded and led AlphaChip, a deep reinforcement learning approach to chip layout published in Nature in 2021, whose layouts have been used for Google TPUs and data center CPUs.234
During time at Anthropic, Mirhoseini contributed to Constitutional AI, a 2022 training approach in which a model critiques and revises its own outputs against a set of principles. Later work on test-time scaling includes Large Language Monkeys, which studied how coverage grows as more samples are drawn at inference time, and the Scaling Intelligence lab's projects include KernelBench, SWiRL, Intelligence Per Watt and the open-source OpenJarvis framework for on-device agents.25
Teaching
Work
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer (external site: arxiv.org)Published research
Co-author, listed second after Noam Shazeer, of the 2017 paper introducing the sparsely-gated Mixture-of-Experts layer.31
Chip Placement with Deep Reinforcement Learning (AlphaChip line of work) (external site: arxiv.org)Published research
First author of "Chip Placement with Deep Reinforcement Learning" (2020). Mirhoseini's Stanford profile lists AlphaChip, a reinforcement learning approach to layout optimization, as past work used in designing chips including Google TPUs.41
Large Language Monkeys: Scaling Inference Compute with Repeated Sampling (external site: arxiv.org)Published research
Co-author (2024) of a study of how repeated sampling scales inference compute for language models, part of the test-time scaling research named on Mirhoseini's Stanford profile.51
Career timeline
2015
Received the Best Thesis Award from the Rice University ECE Department.2
Jan 23, 2017
Co-authored the paper introducing the sparsely-gated Mixture-of-Experts layer.3
Apr 22, 2020
Released the paper on chip placement with deep reinforcement learning that underlies AlphaChip.4
2021
Published the AlphaChip chip-layout work in Nature.2
2022
Contributed to Anthropic's Constitutional AI training approach.2
2023
Began teaching CS229S, Systems for Machine Learning, at Stanford.2
Jul 31, 2024
Co-authored Large Language Monkeys on scaling inference compute with repeated sampling.5
Other documented roles
Roles the sources document but that are not confirmed as current.
- Industry AI lab role, working on Claude among other projects, Anthropic12
- Industry AI lab role, working on Gemini among other projects, Google DeepMind12
- Industry AI lab role, Google Brain (Google)12
Honors and recognition
Related people
More sourced profiles at the same organization, in the same field, or from the same school.
- Andrew NgAlso at Stanford University
- Song HanAlso from Stanford
- Abigail SeeAlso at Google DeepMind
- Brian HieAlso at Stanford University
- Aaron TeoInference and serving
- Abubakar AbidAlso from Stanford
- Carlos GuestrinAlso at Stanford University
- Andrew FeldmanChips and hardware
- Albert GuAlso from Stanford
- Chelsea FinnAlso at Stanford University
- Arjun DesaiAlso from Stanford
- Andrej KarpathyAlso from Stanford
Is this you?
Profiles cover public professional work only, and every line cites a source. If something is out of date, missing or wrong, send a correction with a link to an official page and we will update it. Suggest a correction
Add this badge to your website or GitHub profile README. It links back to this page. It is not an endorsement or a rating.
Markdown (for a README)
[](https://unitedstatesofamericasuperintelligence.com/people/azalia-mirhoseini/)
HTML
<a href="https://unitedstatesofamericasuperintelligence.com/people/azalia-mirhoseini/"><img src="https://unitedstatesofamericasuperintelligence.com/badges/profile.svg" alt="Azalia Mirhoseini — Profile on USASI" height="20"></a>
This profile covers public professional work only, from the sources below. It is not an endorsement or a ranking. To correct or remove it, use the correction form.