How to check an AI claim

To check a claim about AI, first name what kind of claim it is, then find the document that would have to exist if it were true: the evaluation behind a score, the license and released files behind "open source" or "free for commercial use," the provider's own page behind "available now," the developer's published testing behind a safety claim, the official text behind a policy claim, and, for a public company's headcount, a filing with the U.S. Securities and Exchange Commission (SEC). Then check the date and keep your own dated copy. Screenshots, undated pages, "leaked" files, and scores without their setup are leads to trace, not evidence. This page is general information, not legal or investment advice.

Seven kinds of claims and where their evidence lives

Capability or benchmark. "Scores 92% on a coding test." "Reads any document." Look for the model card, the technical report or paper, and the evaluation code if it is published. NIST's draft guidance on automated benchmark evaluations (external site: nvlpubs.nist.gov) (NIST AI 800-2, an initial public draft from January 2026) says evaluators should report key details, including the benchmark version, the number and type of test items, the exact model version, the evaluation protocol, and uncertainty estimates. arXiv says its moderation (external site: info.arxiv.org) "is not a peer-review process," and Hugging Face shows evaluation scores on a model page from the card's metadata (model cards (external site: huggingface.co)) or from result files in the model's repository, which anyone can propose through a pull request (evaluation results (external site: huggingface.co)). Product claims need evidence too: the Federal Trade Commission (FTC) said Workado promoted an AI content detector as "98 percent" accurate, while independent testing showed 53 percent on general-purpose content (April 2025 (external site: ftc.gov)). Its final order requires "competent and reliable evidence" for such claims (August 2025 (external site: ftc.gov)).

Openness, or "open source." Ask what was released (weights, code, information about the training data) and under which license. The Open Source Initiative's Open Source AI Definition 1.0 (external site: opensource.org) requires access to "the preferred form to make modifications to the system": sufficiently detailed information about the training data, the complete source code used to train and run the system, and the model parameters, such as weights, each under terms the OSI approves. By that definition, downloadable weights on their own are not enough. For licenses, the OSI's Open Source Definition (external site: opensource.org) says a license "must not restrict anyone from making use of the program in a specific field of endeavor," and gives use in a business as an example.

License and commercial use. Read the license file, or the license field on the model card, not a summary. On Hugging Face, a custom license is entered as other, with a name and a link. GitHub's documentation on licensing a repository (external site: docs.github.com) says that without a license, default copyright laws apply and "no one may reproduce, distribute, or create derivative works from your work." Commercial use can carry conditions: the Llama 3.1 Community License (external site: github.com) requires "Built with Llama" to be displayed when you distribute the model or a product that contains it, brings in an acceptable use policy, and requires a licensee whose products or services had more than 700 million monthly active users in the calendar month before the Llama 3.1 release date to request a separate license from Meta.

Availability, or "available now." Find the provider's own download page, documentation, or release notes, and read what access requires. A gated model (external site: huggingface.co) on Hugging Face asks you to share your username and email address with its authors. Access may need manual approval, the authors "can decide at any time to block your access," and they can restrict users in European Union countries. "Available" can also mean hosted rather than downloadable. Ollama's gpt-oss page (external site: ollama.com) carries a "cloud" tag and lists gpt-oss:20b-cloud, with no size given, next to gpt-oss:20b at 14GB, and Ollama's documentation says its cloud models (external site: docs.ollama.com) run in Ollama's cloud and do not need to be downloaded.

Safety. Look for the developer's model or system card, its published safety framework, and reports by outside testers. A card is the developer's own account. OpenAI's gpt-oss model card (external site: deploymentsafety.openai.com) (August 5, 2025) says OpenAI engaged outside experts, METR, SecureBio, and Daniel Kang, "to independently review and validate" its malicious fine-tuning methodology. It says they submitted 22 recommendations and OpenAI acted on 11, and an appendix describes its responses. That is still OpenAI's report of the review; how AI developers test models for safety covers what else to look for.

Policy or legal. Go to the official text. The National Archives describes the Federal Register (external site: archives.gov) as "the daily journal of the Federal government," publishing agency regulations, proposed rules, and notices, as well as executive orders and other presidential documents. It calls FederalRegister.gov "our unofficial version." govinfo (external site: govinfo.gov), run by the U.S. Government Publishing Office, provides official publications from all three branches and adds digital signatures to PDF files so users can verify they have not been altered. Finding AI policy and standards sources covers dates, status, and replaced documents.

Company facts. For a public company, start with its filings in the SEC's EDGAR (external site: sec.gov) system, which offers free access and full-text search across more than 20 years of filings. The SEC's investor bulletin on the 10-K (external site: sec.gov), the annual report, says the company writes it and the SEC "neither writes the 10-K nor vouches for its accuracy," though laws prohibit materially false or misleading statements in it. SEC rules (17 CFR 229.101 (external site: govinfo.gov)) ask for a description of human capital resources, "including the number of persons employed by the registrant," to the extent it is material to understanding the business. A company that sells securities under certain exemptions from registration files a Form D notice (external site: sec.gov) within 15 days after the first sale. The form (external site: sec.gov) asks for the total offering amount and the amount sold as of the filing date.

Red flags

Each of these means "trace it further," not "false."

• No date. Without a published or updated date, you cannot tell whether a page describes the current release, license, or policy.
• A screenshot instead of a link. Find the original and read it there; a screenshot cannot show the rest of the page or whether it changed.
• "Leaked" or anonymous. Until the organization or an official source publishes it, treat it as unconfirmed.
• A score without its setup. Without the benchmark version, test items, exact model version, and method, you cannot tell what the number measured.
• "Open source" or "available" with no link. Ask which license covers which parts, and whether access is a download, hosted, gated, or limited to some places or plans.
• Secondhand only. A news story, fact sheet, or post can point to a document; it is not the document.
• Future tense. "Will launch" and "plans to" describe intentions.

Date what you read, and keep a copy

Write down the URL, title, publisher, the date the page shows, and the date you read it. Then pin the version:

• GitHub. Pressing y while viewing a file changes the URL to a permanent link (external site: docs.github.com) that names the exact commit. A branch link can change, because the version at a branch's head "can change as new commits are made."
• Hugging Face. Models are hosted as Git repositories (external site: huggingface.co), with version control, so note the commit you read.
• arXiv. Each replacement of a paper creates a new version (external site: info.arxiv.org). Earlier versions stay available, and you can cite a specific one with the full identifier that includes its version number.
• Federal Register. Note the citation and document number. Executive Order 14434, for example, is 91 FR 63129, document 2026-20321 (API record (external site: federalregister.gov)).

Then save a copy. The Internet Archive's Save Page Now (external site: help.archive.org) gives you "a permanent URL for your page," but it saves a single page, not its outlinks, and some sites prohibit crawling. In the Wayback Machine (external site: help.archive.org), the number in an archived address is a timestamp in the form yyyymmddhhmmss, and the Archive says not every date for every site is complete. Keep your own copy too, in case a page cannot be archived.

Worked example: one post, five claims

Example Lab, its post, and the CodeBench test it names are invented for this page. A post with a screenshot says: "Example Lab's new open-source model beats every rival on CodeBench, is free for commercial use, and is available now. Runs on any laptop!"

1. Split it. Capability ("beats every rival on CodeBench"), openness ("open-source"), license ("free for commercial use"), availability ("available now"), and hardware ("runs on any laptop").
2. Find the source. The screenshot has no link, so you find Example Lab's announcement and model card on its own pages and note their dates.
3. Capability. The card reports a CodeBench score but not the benchmark version, the attempts, or which rival versions were compared. Record it as "reported by Example Lab."
4. Openness and license. The card's license field says other and links to a custom license that bars use in some industries. That conflicts with the field-of-endeavor rule in the Open Source Definition, and "free for commercial use" holds only outside those industries.
5. Availability and hardware. The model is gated with manual approval, and the card gives a memory figure only for a 16-bit version. "Available now" means "available on request," and "any laptop" has no published support; see choosing a local model for your computer.
6. Save it. Archive the card and license, note the commit, and record the date.

Outcome: the score is Example Lab's own report, "open source" and "free for commercial use" do not match the license, "available now" means on request, and "any laptop" is unsupported.

How USASI applies these checks

• Sources. Each catalog claim cites the source that supports it, preferring license files, model and dataset cards, official documentation, release notes, repositories, the organization's own pages, and filings; news reports are used only for the specific claim and labeled (source standards).
• Dates. Each source records its publication date, when the page states one, and the date an editor read it (source standards, dates).
• Openness. Each artifact gets a checklist suited to what it is, and model releases get a disclosure tier from USASI's own rubric v0.2, which is not the OSI's definition or a certification (openness rubric).
• What stays out. The catalog does not publish funding, valuations, staffing numbers, or benchmark scores as facts (source standards), and news items are written only from primary sources (news).
• Unknown. "Unknown" means not assessed or not enough evidence, never a polite "no" (uncertainty).
• Review levels. Pages say when a separate source check was done. USASI does not currently claim individual human review for any page, or a hands-on test for any record or finder entry (review levels).
• Corrections. If USASI gets something wrong, report a correction with the entry, the statement, and a supporting source. Accepted corrections are recorded in the changelog.

What you can do next

• Read how to read an AI evaluation, what open weight and open source actually mean, and how to read a model card.
• For images, audio, and video, see labels, watermarks, and content credentials.
• Look up benchmark, open source AI, license scope, and gated download in the glossary.
• Browse the license guide, the U.S. AI policy tracker, and the source library of primary sources the catalog cites.
• For any claim, write down five things: the claim type, the primary document, its date, the version you read, and where you saved a copy.

Sources

All read on October 13, 2026.

• NIST, Center for AI Standards and Innovation: Practices for Automated Benchmark Evaluations of Language Models (external site: nvlpubs.nist.gov) (NIST AI 800-2, initial public draft, January 2026); CAISSI guidelines page (external site: nist.gov), which lists it as a draft
• Federal Trade Commission: FTC Order Requires Workado to Back Up Artificial Intelligence Detection Claims (external site: ftc.gov) (April 28, 2025); FTC Approves Final Order against Workado, LLC (external site: ftc.gov) (August 28, 2025)
• Open Source Initiative: The Open Source AI Definition, version 1.0 (external site: opensource.org); The Open Source Definition (external site: opensource.org)
• Hugging Face documentation: Model Cards (external site: huggingface.co), Evaluation Results (external site: huggingface.co), Gated models (external site: huggingface.co), Repositories (external site: huggingface.co)
• GitHub Docs: Licensing a repository (external site: docs.github.com); Getting permanent links to files (external site: docs.github.com)
• Meta: Llama 3.1 Community License Agreement (external site: github.com) (version release date July 23, 2024)
• Ollama: gpt-oss library page (external site: ollama.com); Cloud (external site: docs.ollama.com)
• OpenAI: gpt-oss-120b and gpt-oss-20b Model Card (external site: deploymentsafety.openai.com) (August 5, 2025), sections 5.1.1 and Appendix 2
• National Archives, Office of the Federal Register: About the Federal Register (external site: archives.gov) (last reviewed August 8, 2018); Federal Register API record for document 2026-20321 (external site: federalregister.gov)
• U.S. Government Publishing Office: About govinfo (external site: govinfo.gov); 17 CFR 229.101, Code of Federal Regulations (external site: govinfo.gov) (revised as of April 1, 2025)
• U.S. Securities and Exchange Commission: Search Filings (external site: sec.gov); Investor Bulletin: How to Read a 10-K (external site: sec.gov) (September 2011); Filing a Form D notice (external site: sec.gov); Form D (external site: sec.gov)
• arXiv: Moderation (external site: info.arxiv.org); Versions (external site: info.arxiv.org)
• Internet Archive Help Center: Save Pages in the Wayback Machine (external site: help.archive.org); Using the Wayback Machine (external site: help.archive.org)