Ai2 releases OLMo-core 3, its open training library, with mixture-of-experts support

Ai2 released version 3 of OLMo-core, the PyTorch training library behind its Olmo models, on October 1, 2026. The release redesigns the library's mixture-of-experts training system, adding expert and pipeline parallelism, a distributed optimizer, grouped matrix multiplication for experts, and support for the MXFP8 number format; Ai2 says it is designed to scale mixture-of-experts training into the trillion-parameter range. The code is licensed under Apache 2.0, and the repository includes Ai2's official training scripts for Olmo 3.123