Noam Shazeer

Work

• Gemma 3Open weights

  Listed among the sponsors of the Gemma 3 technical report, which describes Gemma 3 as a multimodal addition to the Gemma family of lightweight open models, ranging from 1 to 27 billion parameters.2

  Project page (external site: arxiv.org)

• Transformer (Attention Is All You Need) (external site: arxiv.org)Published research

  Co-author of the 2017 paper that introduced the Transformer architecture; the paper's contribution note credits Shazeer with proposing scaled dot-product attention, multi-head attention, and the parameter-free position representation.3

• Sparsely-gated mixture-of-experts layer (external site: arxiv.org)Published research

  First author of Outrageously Large Neural Networks (2017), which uses a sparsely-gated mixture-of-experts layer to increase model capacity through conditional computation.4

• Tensor2Tensor (external site: github.com)Open-source code

  Co-author of the 2018 Tensor2Tensor paper, which describes the Google Brain library of deep learning models that includes the reference implementation of the Transformer.56

Other documented roles

Roles the sources document but that are not confirmed as current.

• Listed with a Google Brain affiliation on 2017–2018 papers, Google Brain345