Paolo Glorioso
First author of Zyphra's Zamba and Zamba2 technical reports on hybrid state-space language models
Paolo Glorioso is the first author of the technical reports for Zamba (2024), a 7B model that combines a Mamba backbone with a shared attention module, and for the Zamba2 suite (2024) of 1.2B, 2.7B and 7.4B hybrid models, both released with open weights by Zyphra. Glorioso is also listed as an author of Zyphra's 2024 posts on the Zyda pretraining dataset, the Zyphra training cookbook, and the layer-pruning study "The Unreasonable Ineffectiveness of the Deeper Layers".12789
Research
Glorioso is first-listed author of Zyphra's Zamba report, posted in May 2024, and the Zamba2 suite report, posted in November 2024. Zamba applies one shared attention block across a Mamba backbone, and Zamba2 replaces the Mamba1 blocks with Mamba2 blocks and uses two shared attention blocks.124
Glorioso also appears as an author on Zyphra's 2024 releases of Zamba2-small and Zamba2-mini, on the Zyda dataset post, on the Zyphra training cookbook, and, with collaborators from Sequoia Capital, MIT, Meta FAIR and Cisco, on a study of pruning the deeper layers of language models.56789
Work
ZambaOpen weights
First author of the Zamba technical report, which presents a 7B SSM-transformer hybrid trained on openly available datasets and releases its weights and checkpoints.13
Zamba2-7BOpen weights
First author of the Zamba2 suite technical report, which documents the Zamba2 models up to 7.4B parameters and the Zyda-2 pretraining dataset.2
Career timeline
Mar 2024
Listed as an author of Zyphra's post on "The Unreasonable Ineffectiveness of the Deeper Layers".9
Apr 2024
Was first-listed author of Zyphra's release of the Zamba 7B model.3
May 2024
Posted the Zamba technical report on arXiv as first author.1
Jul 2024
Was first-listed author of Zyphra's Zamba2-small (2.7B) release.5
Nov 2024
Posted the Zamba2 suite technical report on arXiv as first author.2
Other documented roles
Roles the sources document but that are not confirmed as current.
Related people
More sourced profiles at the same organization, in the same field, or from the same school.
- Beren MillidgeAlso at Zyphra
- Akshita BhagiaLanguage models
- Hassan ShapourianAlso at Zyphra
- Arthur MenschLanguage models
- Krithik PuthalathAlso at Zyphra
- Barret ZophLanguage models
- Bryan CatanzaroLanguage models
- Ce ZhangTraining systems
- Christopher RéTraining systems
- Clément FarabetLanguage models
- Colin RaffelLanguage models
- Danqi ChenLanguage models
Is this you?
Profiles cover public professional work only, and every line cites a source. If something is out of date, missing or wrong, send a correction with a link to an official page and we will update it. Suggest a correction
Add this badge to your website or GitHub profile README. It links back to this page. It is not an endorsement or a rating.
Markdown (for a README)
[](https://unitedstatesofamericasuperintelligence.com/people/paolo-glorioso/)
HTML
<a href="https://unitedstatesofamericasuperintelligence.com/people/paolo-glorioso/"><img src="https://unitedstatesofamericasuperintelligence.com/badges/profile.svg" alt="Paolo Glorioso — Profile on USASI" height="20"></a>
This profile covers public professional work only, from the sources below. It is not an endorsement or a ranking. To correct or remove it, use the correction form.