Independent project. Not a U.S. government website.

Support our project (opens Buy Me a Coffee)
USASI

Jump to

Start hereFour ways into the catalogCompanies & LabsU.S. AI organizationsOpen Models & ToolsOpen models, software, datasets, and evaluation toolsLearnExplainers and learning pathsFor kidsAI and Super Intelligence, explained for readers 8 to 12Latest newsSourced news itemsPolicy trackerFederal AI laws, orders, and standards

Local AI by memory

AI models you can run on 24 GB of VRAM

With 24 GB of graphics memory (VRAM), you can run 7 documented local configurations in this catalog by their publishers' own memory figures, including OpenAI Whisper turbo on your own computer (about 6 GB), StarCoder2-3B in bfloat16 with Hugging Face Transformers (about 6.3 GB), and llama.cpp server with Gemma 4 12B, QAT Q4_0 GGUF (about 6.7 GB). These are estimates from each publisher's documentation; USASI has not tested them.

Open the finder with 24 GB VRAMChoose another amount

7 configurations fit 24 GB of VRAM

Ordered by the publisher's memory figure, smallest first. This is not a ranking. Each entry names the exact variant the figure is for.

Estimated to fit, not testedOn your computerCommand line or extension

OpenAI Whisper turbo on your own computer

Whisper is OpenAI's open-source speech recognition model, released with a Python package and a command-line tool under the MIT License. Its default turbo model, an optimized version of large-v3, transcribes audio files on your own computer.

Model
Whisper turbo (large-v3-turbo checkpoint)
Variant
Whisper turbo (large-v3-turbo checkpoint) in the openai-whisper package
Documented memory
About 6 GB of graphics memoryPublisher's statement: The Whisper README lists approximate memory requirements per model size, giving about 6 GB of required VRAM for the turbo model; the package maps the name turbo to the large-v3-turbo checkpoint.Sources: Whisper README (external site: raw.githubusercontent.com) · OpenAI (GitHub); whisper/__init__.py (model download table) (external site: raw.githubusercontent.com) · OpenAI (GitHub)
Runtime and tools
Library: openai-whisper (Python package and command line)
License
MIT (weights and code)As recorded, with sources, on the model's catalog record.
Documented for
Windows, macOS, LinuxGraphics hardware: NVIDIA, Processor only

Check before you start

  • Whisper's code and model weights are released under the MIT License and installed with pip; the model weights download from public links built into the package.

Not verified

  • USASI has not tested this configuration.
  • Speed on your hardware is not documented; the README's relative speeds were measured on an A100 GPU.
  • Acceleration on AMD, Intel, or Apple GPUs is not documented in the Whisper repository.

Catalog record: Whisper large-v3-turboOpen in the finder with 24 GB VRAM

Estimated to fit, not testedOn your computerDeveloper setup

StarCoder2-3B in bfloat16 with Hugging Face Transformers

StarCoder2-3B is a 3-billion-parameter code model from the BigCode project, trained on 17 programming languages with a fill-in-the-middle objective. Its model card shows loading it with Hugging Face Transformers in bfloat16 on a CUDA GPU and generating the continuation of a piece of code.

Model
StarCoder2-3B (bigcode/starcoder2-3b)
Variant
starcoder2-3b loaded in torch.bfloat16 with Transformers
Documented memory
About 6.3 GB of graphics memoryPublisher's statement: The model card reports a memory footprint of 6,312.41 MB (about 6.3 GB) after loading the model in torch.bfloat16, as printed by Transformers' get_memory_footprint. This is a footprint the card printed, not a stated minimum; the card states no memory requirement. It also lists 12,624.81 MB in full precision, 3,434.07 MB with 8-bit and 1,994.90 MB with 4-bit bitsandbytes quantization.Source: starcoder2-3b model card (external site: huggingface.co) · BigCode (Hugging Face)
Runtime and tools
Library: Hugging Face Transformers; Library: PyTorch with CUDA
License
BigCode OpenRAIL-M v1 License Agreement (weights); Apache-2.0 (code)As recorded, with sources, on the model's catalog record.
Documented for
Linux, WindowsGraphics hardware: NVIDIA

Check before you start

  • The weights download from Hugging Face without gating under the BigCode OpenRAIL-M v1 license, which carries use restrictions; Transformers is released under Apache 2.0.

Not verified

  • USASI has not tested this configuration.
  • The card's figure is the loaded model's footprint; memory used while generating, which grows with prompt and output length, is not stated.
  • Running on AMD, Intel, or Apple GPUs is not documented on the model card.

Catalog record: StarCoder2-3BOpen in the finder with 24 GB VRAM

Estimated to fit, not testedOn your computerCommand line or extension

llama.cpp server with Gemma 4 12B (QAT Q4_0 GGUF)

llama.cpp's llama-server runs a GGUF model on your own computer and serves a built-in web chat interface and an OpenAI-compatible API. Google publishes Gemma 4 12B as a Q4_0 GGUF built with quantization-aware training and names this download format for local use in llama.cpp.

Model
Gemma 4 12B instruction-tuned, QAT Q4_0 GGUF
Variant
Gemma 4 12B at Q4_0 (4-bit), as in Google's gemma-4-12B-it-qat-q4_0-gguf
Documented memory
About 6.7 GB of graphics memoryPublisher's statement: Google's Gemma 4 documentation lists about 6.7 GB of GPU or TPU memory to load Gemma 4 12B at Q4_0 (4-bit), based on the model weights plus a 20% loading overhead. It says the figure does not include memory for supporting software or the context window, which grows with the length of prompts and responses.Source: Gemma 4 overview (Gemma documentation) (external site: ai.google.dev) · Google
Runtime and tools
Runtime: llama.cpp (llama-server and its web UI)
License
Apache-2.0 (weights)As recorded, with sources, on the model's catalog record.
Documented for
Windows, macOS, LinuxGraphics hardware: NVIDIA, AMD, Apple silicon, Intel, Processor only

Check before you start

  • llama.cpp is open-source software under the MIT License, and Google publishes the Gemma 4 12B QAT GGUF under Apache 2.0 on Hugging Face without gating.

Not verified

  • USASI has not tested this configuration.
  • Google's memory figure excludes the context window; total memory at the context size llama-server chooses on your hardware is not documented.
  • Google's repository also holds an mmproj file for 12B, but llama.cpp's multimodal page lists Gemma 4 E2B, E4B, 26B A4B, and 31B for image input and not 12B, so image input is not established here.
  • Google's table does not name this file; the GGUF in Google's repository is 6,975,879,296 bytes on its own (Hugging Face metadata), slightly more than the table's approximate 6.7 GB.
  • Which pre-built llama.cpp packages include AMD or Intel GPU support is not established here.

Catalog record: Gemma 4 12B UnifiedOpen in the finder with 24 GB VRAM

Estimated to fit, not testedOn your computerDeveloper setup

StarCoder2-15B in 4-bit with Transformers and bitsandbytes

StarCoder2-15B is a 15-billion-parameter code model from the BigCode project, trained on 600+ programming languages with a fill-in-the-middle objective. Its model card shows loading it with Hugging Face Transformers and bitsandbytes 8-bit or 4-bit quantization and generating the continuation of a piece of code on a CUDA GPU.

Model
StarCoder2-15B (bigcode/starcoder2-15b)
Variant
starcoder2-15b loaded with bitsandbytes 4-bit quantization (load_in_4bit) in Transformers
Documented memory
About 9.2 GB of graphics memoryPublisher's statement: The model card reports a memory footprint of 9,224.60 MB (about 9.2 GB) after loading the model with bitsandbytes 4-bit quantization, as printed by Transformers' get_memory_footprint. This is a footprint the card printed, not a stated minimum; the card states no memory requirement. It also lists 16,900.18 MB with 8-bit quantization and 32,251.33 MB in torch.bfloat16.Source: starcoder2-15b model card (external site: huggingface.co) · BigCode (Hugging Face)
Runtime and tools
Library: Hugging Face Transformers; Library: bitsandbytes (4-bit quantization) (external site: huggingface.co); Library: PyTorch with CUDA
License
BigCode OpenRAIL-M v1 License Agreement (weights); Apache-2.0 (code)As recorded, with sources, on the model's catalog record.
Documented for
Linux, WindowsGraphics hardware: NVIDIA

Check before you start

  • The weights download from Hugging Face without gating under the BigCode OpenRAIL-M v1 license, which carries use restrictions; Transformers is Apache 2.0 and bitsandbytes is MIT-licensed.

Not verified

  • USASI has not tested this configuration.
  • The card's figure is the loaded model's footprint; memory used while generating, which grows with prompt and output length, is not stated.
  • The card prints the 4-bit footprint but shows the loading code only for 8-bit, so the 4-bit path is as described there, not checked by USASI.
  • Running on AMD or Intel GPUs is not documented on the model card.

Catalog record: StarCoder2-15BOpen in the finder with 24 GB VRAM

Estimated to fit, not testedOn your computerCommand line or extension

OpenAI Whisper large-v3 on your own computer

Whisper is OpenAI's open-source speech recognition model, released with a Python package and a command-line tool under the MIT License. Its large model, which the package maps to the large-v3 checkpoint, transcribes speech and translates non-English speech into English on your own computer.

Model
Whisper large (large-v3 checkpoint)
Variant
Whisper large (large-v3 checkpoint) in the openai-whisper package
Documented memory
About 10 GB of graphics memoryPublisher's statement: The Whisper README lists approximate memory requirements per model size, giving about 10 GB of required VRAM for the large model; the package maps the name large to the large-v3 checkpoint.Sources: Whisper README (external site: raw.githubusercontent.com) · OpenAI (GitHub); whisper/__init__.py (model download table and device selection) (external site: raw.githubusercontent.com) · OpenAI (GitHub)
Runtime and tools
Library: openai-whisper (Python package and command line)
License
MIT (weights and code); Apache-2.0 (weights)As recorded, with sources, on the model's catalog record.
Documented for
Windows, macOS, LinuxGraphics hardware: NVIDIA, Processor only

Check before you start

  • Whisper's code and model weights are released under the MIT License and installed with pip; the model weights download from public links built into the package.

Not verified

  • USASI has not tested this configuration.
  • Speed on your hardware is not documented; the README's relative speeds were measured on an A100 GPU.
  • Acceleration on AMD, Intel, or Apple GPUs is not documented in the Whisper repository.
  • The ~10 GB figure is stated as VRAM; on a Mac the package picks the CPU (it checks only for CUDA) and uses FP32, and memory use there is not documented.

Catalog record: Whisper large-v3Open in the finder with 24 GB VRAM

Estimated to fit, not testedOn your computerCommand line or extension

llama.cpp server with Gemma 4 26B A4B (QAT Q4_0 GGUF)

llama.cpp's llama-server runs a GGUF model on your own computer and serves a built-in web chat interface and an OpenAI-compatible API. Google publishes Gemma 4 26B A4B, a mixture-of-experts model with about 3.8 billion of its 25.2 billion parameters active per token, as a Q4_0 GGUF built with quantization-aware training, together with a projector file for image input.

Model
Gemma 4 26B A4B instruction-tuned, QAT Q4_0 GGUF
Variant
Gemma 4 26B A4B at Q4_0 (4-bit), as in Google's gemma-4-26B-A4B-it-qat-q4_0-gguf
Documented memory
About 14.4 GB of graphics memoryPublisher's statement: Google's Gemma 4 documentation lists about 14.4 GB of GPU or TPU memory to load Gemma 4 26B A4B at Q4_0 (4-bit), based on the model weights plus a 20% loading overhead. It explains that all 26 billion parameters must be loaded even though only about 4 billion are active per token, and that the figure excludes supporting software and the context window.Source: Gemma 4 overview (Gemma documentation) (external site: ai.google.dev) · Google
Runtime and tools
Runtime: llama.cpp (llama-server and its web UI)
License
Apache-2.0 (weights)As recorded, with sources, on the model's catalog record.
Documented for
Windows, macOS, LinuxGraphics hardware: NVIDIA, AMD, Apple silicon, Intel, Processor only

Check before you start

  • llama.cpp is open-source software under the MIT License, and Google publishes the Gemma 4 26B A4B QAT GGUF under Apache 2.0 on Hugging Face without gating.

Not verified

  • USASI has not tested this configuration.
  • Google's figure covers the model weights only; memory for the image projector and for the context size llama-server chooses is not documented.
  • Google notes its memory numbers may change with the inference tool and environment.
  • Google's table does not name this file; the GGUF in Google's repository is 14,439,363,584 bytes on its own (Hugging Face metadata), about the same as the table's approximate 14.4 GB.
  • Which pre-built llama.cpp packages include AMD or Intel GPU support is not established here.

Catalog record: Gemma 4 26B A4BOpen in the finder with 24 GB VRAM

Estimated to fit, not testedOn your computerCommand line or extension

llama.cpp server with Muse Glimmer 30B (KQuant-17GB Q4_K_M GGUF)

Muse Glimmer is Meta's open-weight 30B model built for agentic work on consumer hardware, including coding agents, tool use, and reading screenshots, charts, and documents. Meta publishes GGUF builds for llama.cpp, including a 4-bit KQuant-17GB text model and a separate perception-encoder file for image input, which llama-server serves with a web chat interface and an OpenAI-compatible API.

Model
Muse Glimmer 30B, KQuant-17GB Q4_K_M GGUF
Variant
Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf, Meta's 4-bit GGUF build
Documented memory
About 24 GB of graphics memoryPublisher's statement: Meta's GGUF model card says the KQuant-17GB build fits 24 GB VRAM, and the Muse Glimmer model card lists 24GB VRAM as the target hardware for that quantization. The GGUF card puts rough total memory, weights plus a working context, at about 17 GB for text only, 19 GB with the perception encoder, and 20 GB with the speculative-decoding drafter as well.Sources: Muse-Glimmer-30B-GGUF model card (external site: huggingface.co) · Meta (Hugging Face); Muse Glimmer model card (meta-models/Muse-Glimmer-30B) (external site: huggingface.co) · Meta (Hugging Face)
Runtime and tools
Runtime: llama.cpp (llama-server and its web UI)
License
Apache-2.0 (weights)As recorded, with sources, on the model's catalog record.
Documented for
Windows, macOS, LinuxGraphics hardware: NVIDIA, Apple silicon, Processor only

Check before you start

  • Meta releases Muse Glimmer's weights, including the GGUF builds, under Apache 2.0, and llama.cpp is open-source software under the MIT License.

Not verified

  • USASI has not tested this configuration.
  • Meta's rough memory totals assume a working context; memory at the long context in the card's server example (131,072 tokens over four slots) is not stated.
  • Meta's card documents CUDA, Apple Metal, and CPU-only builds; AMD and Intel GPUs are not documented for this model in the pages reviewed.
  • How well the model works with a particular coding tool through llama-server is not documented.

Catalog record: Muse Glimmer 30BOpen in the finder with 24 GB VRAM

Stated as system memory, not graphics memory (4)

These figures are recorded as memory or system memory (RAM), not graphics memory, so the finder compares them with system memory and never with VRAM. They are not counted above. Enter your system memory in the finder to compare.

Runs locally, but no memory figure is published (4)

The publisher does not state a memory figure for a specific variant, so USASI does not estimate one and these are not counted above.

How this list is made

Every configuration in the finder records its publisher's own memory statement for one exact variant. A configuration is listed here when the finder, given 24 GB of graphics memory and “on my computer”, finds that every answer is met and that the publisher's figure is at or below 24 GB. It is the same rule the finder uses, not a separate estimate. Only configurations whose documentation lists NVIDIA, AMD or Intel graphics are considered, a graphics-memory figure is compared only with graphics memory, and a figure stated as system memory is never counted against VRAM.

What this does not tell you

  • Speed: how quickly a model responds on your hardware is not documented here.
  • Context length: a publisher's figure can leave out memory for the context window, which grows with longer prompts and responses, and for other software running at the same time. Read each publisher's statement.
  • Operating system, drivers, and graphics support: check each entry's documented platforms and graphics hardware.
  • Testing: USASI has not run these configurations unless an entry says it was tested.

To estimate memory for a model and precision yourself, use the model size calculator, or read understanding inference hardware.

Choose another amount

The number in brackets is how many documented configurations fit each amount.

All amounts · Running AI on your own hardware · Find AI for my needs

Support Us

Help keep USASI useful.

Leave a tip on Buy Me a Coffee (external site)

Optional. No USASI account required. Payment takes place on the linked provider’s website (Buy Me a Coffee).

About supporting this project