KempnerForge
Project record
Maintained by Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University125
KempnerForge is a PyTorch-native framework from Harvard's Kempner Institute for fault-tolerant distributed training of foundation models on AI clusters. It trains decoder-only Transformers and mixture-of-experts models with FSDP2, tensor, expert, and pipeline parallelism and FP8 mixed precision, and it includes asynchronous checkpointing with auto-resume, SLURM preemption handling, activation-extraction hooks for interpretability, and vision-language model training.1
- Repository: Repository (KempnerInstitute/KempnerForge) (external site: github.com)
- Documentation: Documentation (external site: kempnerinstitute.github.io)
- License: LICENSE (MIT) (external site: github.com)
- Release notes: Releases (external site: github.com)
Availability and license
Overall availability
Source code is public on GitHub and is installed from a clone with uv.1
Availability is separate from permission: read the license before using or redistributing.
Component reuse rights
- weights
- Unknown — no complete fact-level rights review
- code
- Reviewed qualifying license recorded — check scope and conditions
- data
- Unknown — no complete fact-level rights review
- documentation
- Unknown — no complete fact-level rights review
No complete system-rights review is recorded for this release.
Public materials checklist
| Item | Status | Notes and evidence |
|---|---|---|
| Source codeIs the source code publicly readable? | Public | Public GitHub repository under the MIT License.12 |
| DocumentationIs user documentation published? | Public | A documentation site built from the repository covers getting started, how-to guides (including an end-to-end training run), architecture, and API reference, and the repository includes example notebooks.31 |
| InstallationAre installation instructions or packages publicly available? | Public | The README's quick start installs dependencies with "uv sync" and runs training scripts on one GPU, several GPUs with torchrun, or SLURM.1 |
| Supported platformsAre supported operating systems or hardware documented? | Partial | The README lists Python 3.12 or later and PyTorch 2.4 or later with CUDA as prerequisites, and includes SLURM launch scripts. No broader hardware or operating-system support list was found.1 |
| Release statusAre versioned releases published? | Public | The GitHub releases page lists v0.1.0, released April 20, 2026.4 |
What it is useful for
The README lists scaling-law experiments, mechanistic interpretability, sparse (mixture-of-experts) architecture research, optimizer and learning-rate-schedule comparisons, long-running jobs on shared SLURM clusters, and extracting model representations to compare with neural recordings in NeuroAI research.1
Run and use notes
- The README's quick start runs a single-GPU debug configuration, a four-GPU FSDP run with torchrun, and a single-node SLURM job, each driven by a TOML configuration file.1
Organization context
U.S. eligibility
Eligible · basis: U.S.-governed project
KempnerForge is published in the Kempner Institute's GitHub organization, and its MIT License names the Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University, as copyright holder. Harvard University is a 501(c)(3) school in Cambridge, Massachusetts, according to the IRS exempt-organization extract.1256
Sources
This listing is not an endorsement, a safety assessment, or a federal approval.