Organization
MLCommons
Standards body · Open-source steward · Dover, Delaware (address listed on the About page)
MLCommons is an AI engineering consortium whose members include companies, academics, and nonprofits. It develops benchmarks, including the MLPerf performance suites and the AILuminate safety and security benchmarks, as well as open datasets and tools such as Croissant and MLCube. It launched in December 2020.12
Key facts
What is and isn’t public
Publishes MLPerf reference implementations and the ModelBench benchmarking framework in public GitHub repositories under Apache 2.0, plus 1,200-prompt AILuminate demo sets in English and French under CC BY 4.0. AILuminate's official test prompts are kept hidden, and its full practice prompt sets are offered to MLCommons members.5678
Products
No product records yet.
Related open artifacts
Evaluation tools
- AILuminateEvaluation toolAvailability: Partial
- MLPerfEvaluation toolAvailability: Public
Notable documented facts
- MLPerf is a registered trademark of MLCommons, and submitting benchmark results requires a signed Contributor License Agreement.2
- Under the MLPerf submission rules, submitting organizations review one another's results during a confidential review period before MLCommons publishes all results at once; after publication, code and results are free to use under the MLPerf Terms of Use.3
- The AILuminate benchmarks are developed by the MLCommons AI Risk & Reliability working group, which at the v1.0 release included researchers from universities, civil society representatives, and engineers from technology companies.4
U.S. eligibility
Eligible · basis: U.S. nonprofit or lab
MLCommons Association lists a Dover, Delaware address on its About page, and the IRS exempt-organization extract for Delaware lists MLCommons Association at that address as a 501(c)(6) organization. Its members come from many countries; this assessment concerns the association itself.19
Sources
This listing is not an endorsement or a federal approval.