Nicholas Carlini
Research scientist at Anthropic working on machine learning security; formerly at Google Brain and DeepMind
Nicholas Carlini is a research scientist at Anthropic who works at the intersection of machine learning and computer security, studying what harmful things could be done with, or to, language models. Carlini was previously a research scientist at Google Brain from 2018 to 2023 and at DeepMind from 2023 to 2025. Carlini is first author of "Extracting Training Data from Large Language Models" (2020) and "Stealing Part of a Production Language Model" (2024), and publishes Yet Another Applied LLM Benchmark, a GPL-3.0 test suite built from tasks Carlini has asked language models to do.1234
Work
Extracting Training Data from Large Language Models (external site: arxiv.org)Published research
First author of the 2020 paper showing that individual training examples can be recovered from a language model by querying it.2
Stealing Part of a Production Language Model (external site: arxiv.org)Published research
First author of the 2024 paper on a model-stealing attack that recovers precise information from black-box production language models.3
Yet Another Applied LLM Benchmark (external site: github.com)Open-source code
Author of this GPL-3.0 benchmark, which uses a small dataflow domain-specific language to define tests derived from tasks Carlini has asked language models to perform.4
Other documented roles
Roles the sources document but that are not confirmed as current.
- Research scientist (2023–2025), DeepMind1
- Research scientist (2018–2023), Google Brain1
This profile covers public professional work only, from the sources below. It is not an endorsement or a ranking. To correct or remove it, use the correction form.