Compare
Model releases view
How to read this comparison
Rows are organization records with a documented hosted-model product or at least one model record. Hosted model products come from sourced product records on each organization page (a product indicator, not a release count). All other columns count model records in the Open Models & Tools directory.
Tier columns use USASI rubric v0.2 and are cumulative, so they overlap: every reviewed open systems release is also counted as open-stack and open-weight. Do not add them together. A release listing two organizations appears in both rows, so columns are not summed.
Counts describe catalog coverage, not capability. Open-weight means publicly downloadable weights, not unrestricted use. Select a count to inspect its records.
Choose up to four organizations for a focused comparison. With none selected, all organizations are shown in name order.
Download this view:
Column definitions and counting rules
- Model families
- Family overview records. Families summarize releases and are never counted as releases.
- Model releases
- Specific releases assessed in this catalog.
- Open-weight
- Releases with publicly downloadable weights. Includes every open-stack and reviewed open-system release.
- Open-stack
- Subset of open-weight releases that also publish code, recipe, and training-data information. Includes reviewed open-system releases.
- Open system (reviewed)
- Open-stack releases with complete information and an explicit sourced system-rights review; not OSI certification.
- Training code public
- Releases whose training code is documented as public.
- Training-data info
- Releases whose training data is public, or whose composition is documented (public or partial).
- Evaluation materials public
- Releases whose evaluation code or prompts are published, so others can re-run the evaluations. Results-only releases are not counted.
- Restricted weights
- Releases whose weights are available only by request or approval, or to some users. Not counted as open-weight.
- Weights unassessed
- Releases whose weight availability is Unknown in this catalog.
- Zero versus unknown
- 0 means no matching published record in this catalog — not proof that nothing exists. “Weights unassessed” counts releases whose weight availability is Unknown.
- What is excluded
- Drafts and archived historical records are excluded from every count.
Software, data & evaluation view
Each kind of artifact is compared on its own checklist. Status is shown with an icon and text; Unknown means unassessed or insufficient evidence.
Frameworks and runtimes
| Name | Maintainer | Availability | License | Source code | Documentation | Installation | Supported platforms | Release status | Reviewed |
|---|---|---|---|---|---|---|---|---|---|
| Agent Development Kit (ADK) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 | |
| Agent2Agent (A2A) Protocol | Agent2Agent project (Linux Foundation; Agentic AI Foundation project) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| AGENTS.md | AGENTS.md a Series of LF Projects, LLC (Agentic AI Foundation project) | Public | MIT | Source code: Public | Documentation: Public | Installation: Not applicable | Supported platforms: Not applicable | Release status: Not applicable | Oct 1, 2026 |
| AI SDK | Vercel | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| AnythingLLM | Mintplex Labs | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| AutoGen | Microsoft | Public | MIT, CC-BY-4.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| AWQ (Activation-aware Weight Quantization) | MIT HAN Lab | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Not public | Oct 1, 2026 |
| bitnet.cpp (BitNet inference framework) | Microsoft | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Partial | Oct 1, 2026 |
| Chroma (open-source database) | Chroma | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Unknown | Release status: Public | Sep 29, 2026 |
| Cline (open-source coding agent) | Cline Bot Inc. | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| Codex CLI | OpenAI | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| Core ML Tools (coremltools) | Apple | Public | BSD-3-Clause | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| DeepSpeed | PyTorch Foundation (hosted project), DeepSpeed Technical Steering Committee (project committers) | Public | Apache-2.0, CC-BY-4.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| Diffusers | Hugging Face | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| Dioptra | National Institute of Standards and Technology (NIST) | Public | CC-BY-4.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Partial | Release status: Public | Oct 1, 2026 |
| Docling | LF AI & Data Foundation (hosted project), IBM Research (originating team) | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| DSPy | Stanford NLP Group | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Partial | Release status: Public | Sep 29, 2026 |
| ExecuTorch | Meta | Public | BSD-3-Clause | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| Faiss | Meta (Fundamental AI Research) | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| FastVideo | Hao AI Lab, UC San Diego | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| Gemini CLI | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 | |
| goose | goose, a Series of LF Projects, LLC (Agentic AI Foundation project) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| GPT4All | Nomic AI | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| Hermes Agent | Nous Research | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| HydraGNN | Oak Ridge National Laboratory | Public | BSD-3-Clause | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Partial | Release status: Public | Oct 1, 2026 |
| Inspect-Hawk (Hawk) | METR (Model Evaluation & Threat Research) | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 2, 2026 |
| JAX | JAX core team (Google open-source project) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| KempnerForge | Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Partial | Release status: Public | Oct 1, 2026 |
| Keras | Keras team at Google | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| LangChain (framework) | LangChain Inc. | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| LangGraph | LangChain Inc. | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| LBANN (Livermore Big Artificial Neural Network Toolkit) | Lawrence Livermore National Laboratory (LBANN project) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Partial | Release status: Public | Oct 1, 2026 |
| Lemonade | Lemonade project (sponsored and backed by AMD, with community maintainers) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| Liger Kernel | Public | BSD-2-Clause | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 | |
| LiteRT | Google (Google AI Edge) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| llama.cpp | ggml team at Hugging Face (ggml.ai, acquired by Hugging Face in 2026), with the ggml-org community | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| llamafile | Mozilla.ai | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| LlamaIndex (framework) | LlamaIndex, Inc. | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Unknown | Release status: Public | Sep 29, 2026 |
| llm-d | Cloud Native Computing Foundation (Linux Foundation), Sandbox project, Red Hat (launched the project; MAINTAINERS.md lists Red Hat staff in project leadership and as community managers), Google (MAINTAINERS.md lists Google staff in project leadership), IBM (IBM Research is a founding contributor; MAINTAINERS.md lists IBM staff in project leadership and as a community manager) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| LLMRouter | U Lab, University of Illinois Urbana-Champaign | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Partial | Release status: Partial | Oct 1, 2026 |
| MarkItDown | Microsoft | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Partial | Release status: Public | Oct 1, 2026 |
| MAX (Modular) | Modular | Public | Apache-2.0 WITH LLVM-exception, Modular Community License (MAX package usage and distribution) | Source code: Partial | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| MLC LLM | MLC team (MLC open community), CMU Catalyst (lists MLC LLM among its own research projects) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Partial | Sep 29, 2026 |
| MLflow | MLflow Project, a Series of LF Projects, LLC (Linux Foundation), Databricks (creator) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| MLX | Apple (machine learning research) | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| Model Context Protocol (MCP) | Model Context Protocol a Series of LF Projects, LLC (Agentic AI Foundation project) | Public | Apache-2.0, MIT, CC-BY-4.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| Mojo | Modular | Public | Apache-2.0 WITH LLVM-exception | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| NeuroMANCER | Pacific Northwest National Laboratory | Public | BSD-style license (Battelle Memorial Institute) | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Partial | Release status: Public | Oct 1, 2026 |
| NVIDIA cuOpt | NVIDIA | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| NVIDIA Dynamo | NVIDIA | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| NVIDIA Isaac Lab | NVIDIA | Public | BSD-3-Clause, Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| NVIDIA NeMo Framework | NVIDIA | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| Ollama | Ollama Inc. | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| OLMo-core | Ai2 | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Partial | Release status: Public | Oct 8, 2026 |
| ONNX (Open Neural Network Exchange) | ONNX project (hosted by the LF AI & Data Foundation) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| ONNX Runtime | Microsoft | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| Open WebUI | Open WebUI, Inc. | Public | Open WebUI License (BSD 3-Clause terms plus a branding clause), BSD-3-Clause, MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Partial | Release status: Public | Oct 1, 2026 |
| OpenAI Agents SDK | OpenAI | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| OpenAI MCP Extensions | OpenAI | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 4, 2026 |
| OpenClaw | OpenClaw Foundation | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| OpenHands | All Hands AI (OpenHands) | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| OpenVINO | Intel | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| OpenXLA | OpenXLA project maintainers (Google-led technical leadership) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Unknown | Sep 29, 2026 |
| PaperQA (paper-qa) | FutureHouse | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Partial | Release status: Public | Oct 1, 2026 |
| PyTorch | PyTorch Foundation | Public | BSD-3-Clause | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| PyTorch Lightning | Lightning AI | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| Qualcomm AI Hub Models | Qualcomm Technologies, Inc. | Public | BSD-3-Clause | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| Ray | PyTorch Foundation (hosted project) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| ROCm | AMD | Public | MIT, Apache-2.0, Apache-2.0 WITH LLVM-exception, NCSA, BSD-2-Clause, GNU General Public License v3.0 (ROCgdb), GPL 2.0 with Linux-syscall-note (ROCK-Kernel-Driver), Custom licenses (hipCUB, RCCL) | Source code: Partial | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 8, 2026 |
| Sentence Transformers | Hugging Face | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Partial | Release status: Public | Oct 8, 2026 |
| SGLang | LMSYS (hosting organization), RadixArk (company described as a maintainer in a post on the LMSYS blog) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| smolagents | Hugging Face | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| Spark (sparkjs) | World Labs | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| Spec Kit | GitHub | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| Strands Agents | Amazon Web Services (AWS) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| SynthID Bio | Google DeepMind | Public | Apache-2.0, MIT, CC-BY-4.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Partial | Release status: Not public | Oct 4, 2026 |
| SynthID Text | Google DeepMind | Public | Apache-2.0, CC-BY-4.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Partial | Release status: Public | Oct 8, 2026 |
| T3 Code | T3 Tools, Inc. (Ping Labs) | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 18, 2026 |
| TensorFlow | Google (TensorFlow team) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| TensorRT-LLM | NVIDIA | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| tiktoken | OpenAI | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 8, 2026 |
| tinygrad | tiny corp | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
| Transformer Explainer | Polo Club of Data Science, Georgia Tech | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Partial | Release status: Partial | Oct 1, 2026 |
| Transformers | Hugging Face | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| TRL (Transformers Reinforcement Learning) | Hugging Face | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Unknown | Release status: Public | Sep 29, 2026 |
| Unsloth (open-source library) | Unsloth | Public | Apache-2.0, AGPL-3.0-only | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| Vet (Verify Everything) | Imbue | Public | AGPL-3.0-only | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Partial | Release status: Public | Oct 2, 2026 |
| vLLM | PyTorch Foundation (hosted project), vLLM core maintainers (project Technical Steering Committee) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Sep 29, 2026 |
| Waymax | Waymo | Public | Waymax License Agreement for Non-Commercial Use | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Not public | Sep 29, 2026 |
| Weights & Biases Python SDK (wandb) | Weights & Biases (a CoreWeave company) | Public | MIT | Source code: Public | Documentation: Public | Installation: Public | Supported platforms: Public | Release status: Public | Oct 1, 2026 |
Research stacks
| Name | Maintainer | Availability | License | Source code | Documentation | Training code | Data information | Reproducibility | Reviewed |
|---|---|---|---|---|---|---|---|---|---|
| GPT-NeoX | EleutherAI | Public | Apache-2.0 | Source code: Public | Documentation: Public | Training code: Public | Data information: Public | Reproducibility: Public | Sep 29, 2026 |
| Marin | Open Athena (Open Athena AI Foundation Inc.), Stanford Center for Research on Foundation Models (CRFM) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Training code: Public | Data information: Public | Reproducibility: Public | Oct 1, 2026 |
| MaxText | Google (AI Hypercomputer GitHub organization) | Public | Apache-2.0 | Source code: Public | Documentation: Public | Training code: Public | Data information: Public | Reproducibility: Unknown | Sep 29, 2026 |
| Megatron-LM and Megatron Core | NVIDIA | Public | BSD-3-Clause | Source code: Public | Documentation: Public | Training code: Public | Data information: Public | Reproducibility: Unknown | Sep 29, 2026 |
| openpi | Physical Intelligence | Public | Apache-2.0 | Source code: Public | Documentation: Public | Training code: Public | Data information: Partial | Reproducibility: Partial | Sep 29, 2026 |
| prime-rl | Prime Intellect | Public | Apache-2.0 | Source code: Public | Documentation: Public | Training code: Public | Data information: Public | Reproducibility: Partial | Sep 29, 2026 |
Datasets
| Name | Maintainer | Availability | License | Access | Provenance | Documentation | Licensing | Stated limitations | Reviewed |
|---|---|---|---|---|---|---|---|---|---|
| Common Crawl corpus | Common Crawl Foundation | Public | Common Crawl Terms of Use (limited, non-exclusive license) | Access: Public | Provenance: Public | Documentation: Public | Licensing: Partial | Stated limitations: Public | Sep 29, 2026 |
| Dolma | Ai2 (Allen Institute for AI) | Public | ODC-By-1.0, Apache-2.0 | Access: Public | Provenance: Public | Documentation: Public | Licensing: Public | Stated limitations: Public | Sep 29, 2026 |
| Epoch AI Data on AI Models | Epoch AI | Public | CC-BY-4.0 | Access: Public | Provenance: Partial | Documentation: Public | Licensing: Public | Stated limitations: Partial | Oct 2, 2026 |
| Essential-Web v1.0 | Essential AI | Public | ODC-By-1.0 | Access: Public | Provenance: Public | Documentation: Public | Licensing: Public | Stated limitations: Unknown | Sep 29, 2026 |
| FineWeb | Hugging Face (FineData, HuggingFaceFW) | Public | ODC-By-1.0, Apache-2.0 | Access: Public | Provenance: Public | Documentation: Public | Licensing: Public | Stated limitations: Public | Sep 29, 2026 |
| Nemotron-CC | NVIDIA | Partial | Common Crawl Terms of Use (original Nemotron-CC release), NVIDIA Data Agreement for Model Training (Nemotron-CC-v2 and v2.1) | Access: Partial | Provenance: Public | Documentation: Public | Licensing: Public | Stated limitations: Public | Sep 29, 2026 |
| RedPajama | Together AI (Together Computer, Inc.) | Public | Apache-2.0 | Access: Public | Provenance: Public | Documentation: Public | Licensing: Partial | Stated limitations: Unknown | Sep 29, 2026 |
| The Pile | EleutherAI | Partial | MIT | Access: Partial | Provenance: Public | Documentation: Public | Licensing: Partial | Stated limitations: Public | Sep 29, 2026 |
| Waymo Open Dataset | Waymo | Partial | Waymo Dataset License Agreement for Non-Commercial Use (March 2025), Apache-2.0, BSD 3-clause copyright license plus a limited patent license (wdl_limited folders) | Access: Partial | Provenance: Public | Documentation: Public | Licensing: Public | Stated limitations: Public | Sep 29, 2026 |
Evaluation tools
| Name | Maintainer | Availability | License | Code | Tasks / data | Methodology | Reproducibility | Limitations | Reviewed |
|---|---|---|---|---|---|---|---|---|---|
| AILuminate | MLCommons (AI Risk & Reliability working group) | Partial | CC-BY-4.0, Apache-2.0 | Code: Public | Tasks / data: Partial | Methodology: Public | Reproducibility: Partial | Limitations: Public | Sep 29, 2026 |
| ARC-AGI | ARC Prize Foundation | Partial | Apache-2.0, MIT | Code: Public | Tasks / data: Partial | Methodology: Public | Reproducibility: Partial | Limitations: Partial | Sep 29, 2026 |
| Berkeley Function Calling Leaderboard (BFCL) | Gorilla project, UC Berkeley | Public | Apache-2.0 | Code: Public | Tasks / data: Public | Methodology: Public | Reproducibility: Public | Limitations: Unknown | Sep 29, 2026 |
| BigCodeBench | BigCode project (open scientific collaboration), Hugging Face (co-leads the BigCode steering committee), ServiceNow (co-leads the BigCode steering committee) | Public | Apache-2.0 | Code: Public | Tasks / data: Public | Methodology: Public | Reproducibility: Public | Limitations: Public | Sep 29, 2026 |
| Harvey LAB (Legal Agent Benchmark) | Harvey | Public | MIT | Code: Public | Tasks / data: Public | Methodology: Public | Reproducibility: Public | Limitations: Partial | Oct 2, 2026 |
| HELM (Holistic Evaluation of Language Models) | Stanford Center for Research on Foundation Models (CRFM) | Public | Apache-2.0 | Code: Public | Tasks / data: Partial | Methodology: Public | Reproducibility: Public | Limitations: Partial | Sep 29, 2026 |
| HumanEval | OpenAI | Public | MIT | Code: Public | Tasks / data: Public | Methodology: Public | Reproducibility: Public | Limitations: Partial | Sep 29, 2026 |
| Humanity's Last Exam (HLE) | Center for AI Safety, Scale AI | Partial | MIT | Code: Public | Tasks / data: Partial | Methodology: Public | Reproducibility: Public | Limitations: Public | Sep 29, 2026 |
| LM Evaluation Harness | EleutherAI | Public | MIT | Code: Public | Tasks / data: Public | Methodology: Public | Reproducibility: Public | Limitations: Partial | Sep 29, 2026 |
| MLPerf | MLCommons | Public | Apache-2.0 | Code: Public | Tasks / data: Public | Methodology: Public | Reproducibility: Public | Limitations: Partial | Sep 29, 2026 |
| OpenAI Evals | OpenAI | Public | MIT | Code: Public | Tasks / data: Public | Methodology: Public | Reproducibility: Public | Limitations: Partial | Sep 29, 2026 |
| Petri (Inspect Petri) | Meridian Labs | Public | MIT | Code: Public | Tasks / data: Public | Methodology: Public | Reproducibility: Public | Limitations: Partial | Oct 1, 2026 |
| SWE-sweep | Meta | Public | MIT | Code: Public | Tasks / data: Public | Methodology: Public | Reproducibility: Public | Limitations: Public | Oct 4, 2026 |
| Terminal-Bench | Stanford University, Laude Institute, Harbor (harbor-framework project) | Public | Apache-2.0 | Code: Public | Tasks / data: Public | Methodology: Public | Reproducibility: Public | Limitations: Public | Sep 29, 2026 |
| WorkflowEvals (TypeSafe AI) | TypeSafe AI | Public | Apache-2.0 | Code: Public | Tasks / data: Public | Methodology: Public | Reproducibility: Public | Limitations: Partial | Oct 1, 2026 |
| τ-bench (tau2-bench) | Sierra | Public | MIT | Code: Public | Tasks / data: Public | Methodology: Public | Reproducibility: Public | Limitations: Partial | Sep 29, 2026 |
Catalog coverage
Totals of published records. Organization records include research units and subsidiaries, which are also shown separately so a parent and its unit are never presented as two independent companies. Model families are counted separately from releases.
| Record type | Count | What is counted |
|---|---|---|
| Organization recordsEvery published organization record, including research units and subsidiaries listed separately. | 152 | Every published organization record, including research units and subsidiaries listed separately. |
| Top-level organizationsOrganization records without a parent organization in this catalog. | 140 | Organization records without a parent organization in this catalog. |
| Units and subsidiariesOrganization records that belong to a parent listed in this catalog. Not independent companies. | 12 | Organization records that belong to a parent listed in this catalog. Not independent companies. |
| Model familiesOverview records. Not added to release counts. | 54 | Overview records. Not added to release counts. |
| Model releasesSpecific releases assessed. | 101 | Specific releases assessed. |
| FrameworksSoftware libraries for building or training models. | 63 | Software libraries for building or training models. |
| RuntimesSoftware for serving or running models. | 27 | Software for serving or running models. |
| Research stacksReference training or research codebases. | 6 | Reference training or research codebases. |
| DatasetsPublished datasets. | 9 | Published datasets. |
| Evaluation toolsBenchmarks and evaluation harnesses. | 16 | Benchmarks and evaluation harnesses. |
Support Us
Help keep USASI useful.
Optional. No USASI account required. Payment takes place on the linked provider’s website (Buy Me a Coffee).
About supporting this project