With 16 GB of graphics memory (VRAM), you can run 6 documented local configurations in this catalog by their publishers' own memory figures, including OpenAI Whisper turbo on your own computer (about 6 GB), StarCoder2-3B in bfloat16 with Hugging Face Transformers (about 6.3 GB), and llama.cpp server with Gemma 4 12B, QAT Q4_0 GGUF (about 6.7 GB). These are estimates from each publisher's documentation; USASI has not tested them.
Ordered by the publisher's memory figure, smallest first. This is not a ranking. Each entry names the exact variant the figure is for.
Estimated to fit, not testedOn your computerCommand line or extension
OpenAI Whisper turbo on your own computer
Whisper is OpenAI's open-source speech recognition model, released with a Python package and a command-line tool under the MIT License. Its default turbo model, an optimized version of large-v3, transcribes audio files on your own computer.
Windows, macOS, LinuxGraphics hardware: NVIDIA, Processor only
Check before you start
Whisper's code and model weights are released under the MIT License and installed with pip; the model weights download from public links built into the package.
Not verified
USASI has not tested this configuration.
Speed on your hardware is not documented; the README's relative speeds were measured on an A100 GPU.
Acceleration on AMD, Intel, or Apple GPUs is not documented in the Whisper repository.
Estimated to fit, not testedOn your computerDeveloper setup
StarCoder2-3B in bfloat16 with Hugging Face Transformers
StarCoder2-3B is a 3-billion-parameter code model from the BigCode project, trained on 17 programming languages with a fill-in-the-middle objective. Its model card shows loading it with Hugging Face Transformers in bfloat16 on a CUDA GPU and generating the continuation of a piece of code.
starcoder2-3b loaded in torch.bfloat16 with Transformers
Documented memory
About 6.3 GB of graphics memoryPublisher's statement: The model card reports a memory footprint of 6,312.41 MB (about 6.3 GB) after loading the model in torch.bfloat16, as printed by Transformers' get_memory_footprint. This is a footprint the card printed, not a stated minimum; the card states no memory requirement. It also lists 12,624.81 MB in full precision, 3,434.07 MB with 8-bit and 1,994.90 MB with 4-bit bitsandbytes quantization.Source: starcoder2-3b model card (external site: huggingface.co) · BigCode (Hugging Face)
The weights download from Hugging Face without gating under the BigCode OpenRAIL-M v1 license, which carries use restrictions; Transformers is released under Apache 2.0.
Not verified
USASI has not tested this configuration.
The card's figure is the loaded model's footprint; memory used while generating, which grows with prompt and output length, is not stated.
Running on AMD, Intel, or Apple GPUs is not documented on the model card.
Estimated to fit, not testedOn your computerCommand line or extension
llama.cpp server with Gemma 4 12B (QAT Q4_0 GGUF)
llama.cpp's llama-server runs a GGUF model on your own computer and serves a built-in web chat interface and an OpenAI-compatible API. Google publishes Gemma 4 12B as a Q4_0 GGUF built with quantization-aware training and names this download format for local use in llama.cpp.
Gemma 4 12B at Q4_0 (4-bit), as in Google's gemma-4-12B-it-qat-q4_0-gguf
Documented memory
About 6.7 GB of graphics memoryPublisher's statement: Google's Gemma 4 documentation lists about 6.7 GB of GPU or TPU memory to load Gemma 4 12B at Q4_0 (4-bit), based on the model weights plus a 20% loading overhead. It says the figure does not include memory for supporting software or the context window, which grows with the length of prompts and responses.Source: Gemma 4 overview (Gemma documentation) (external site: ai.google.dev) · Google
Windows, macOS, LinuxGraphics hardware: NVIDIA, AMD, Apple silicon, Intel, Processor only
Check before you start
llama.cpp is open-source software under the MIT License, and Google publishes the Gemma 4 12B QAT GGUF under Apache 2.0 on Hugging Face without gating.
Not verified
USASI has not tested this configuration.
Google's memory figure excludes the context window; total memory at the context size llama-server chooses on your hardware is not documented.
Google's repository also holds an mmproj file for 12B, but llama.cpp's multimodal page lists Gemma 4 E2B, E4B, 26B A4B, and 31B for image input and not 12B, so image input is not established here.
Google's table does not name this file; the GGUF in Google's repository is 6,975,879,296 bytes on its own (Hugging Face metadata), slightly more than the table's approximate 6.7 GB.
Which pre-built llama.cpp packages include AMD or Intel GPU support is not established here.
Estimated to fit, not testedOn your computerDeveloper setup
StarCoder2-15B in 4-bit with Transformers and bitsandbytes
StarCoder2-15B is a 15-billion-parameter code model from the BigCode project, trained on 600+ programming languages with a fill-in-the-middle objective. Its model card shows loading it with Hugging Face Transformers and bitsandbytes 8-bit or 4-bit quantization and generating the continuation of a piece of code on a CUDA GPU.
starcoder2-15b loaded with bitsandbytes 4-bit quantization (load_in_4bit) in Transformers
Documented memory
About 9.2 GB of graphics memoryPublisher's statement: The model card reports a memory footprint of 9,224.60 MB (about 9.2 GB) after loading the model with bitsandbytes 4-bit quantization, as printed by Transformers' get_memory_footprint. This is a footprint the card printed, not a stated minimum; the card states no memory requirement. It also lists 16,900.18 MB with 8-bit quantization and 32,251.33 MB in torch.bfloat16.Source: starcoder2-15b model card (external site: huggingface.co) · BigCode (Hugging Face)
The weights download from Hugging Face without gating under the BigCode OpenRAIL-M v1 license, which carries use restrictions; Transformers is Apache 2.0 and bitsandbytes is MIT-licensed.
Not verified
USASI has not tested this configuration.
The card's figure is the loaded model's footprint; memory used while generating, which grows with prompt and output length, is not stated.
The card prints the 4-bit footprint but shows the loading code only for 8-bit, so the 4-bit path is as described there, not checked by USASI.
Running on AMD or Intel GPUs is not documented on the model card.
Estimated to fit, not testedOn your computerCommand line or extension
OpenAI Whisper large-v3 on your own computer
Whisper is OpenAI's open-source speech recognition model, released with a Python package and a command-line tool under the MIT License. Its large model, which the package maps to the large-v3 checkpoint, transcribes speech and translates non-English speech into English on your own computer.
Windows, macOS, LinuxGraphics hardware: NVIDIA, Processor only
Check before you start
Whisper's code and model weights are released under the MIT License and installed with pip; the model weights download from public links built into the package.
Not verified
USASI has not tested this configuration.
Speed on your hardware is not documented; the README's relative speeds were measured on an A100 GPU.
Acceleration on AMD, Intel, or Apple GPUs is not documented in the Whisper repository.
The ~10 GB figure is stated as VRAM; on a Mac the package picks the CPU (it checks only for CUDA) and uses FP32, and memory use there is not documented.
Estimated to fit, not testedOn your computerCommand line or extension
llama.cpp server with Gemma 4 26B A4B (QAT Q4_0 GGUF)
llama.cpp's llama-server runs a GGUF model on your own computer and serves a built-in web chat interface and an OpenAI-compatible API. Google publishes Gemma 4 26B A4B, a mixture-of-experts model with about 3.8 billion of its 25.2 billion parameters active per token, as a Q4_0 GGUF built with quantization-aware training, together with a projector file for image input.
Gemma 4 26B A4B at Q4_0 (4-bit), as in Google's gemma-4-26B-A4B-it-qat-q4_0-gguf
Documented memory
About 14.4 GB of graphics memoryPublisher's statement: Google's Gemma 4 documentation lists about 14.4 GB of GPU or TPU memory to load Gemma 4 26B A4B at Q4_0 (4-bit), based on the model weights plus a 20% loading overhead. It explains that all 26 billion parameters must be loaded even though only about 4 billion are active per token, and that the figure excludes supporting software and the context window.Source: Gemma 4 overview (Gemma documentation) (external site: ai.google.dev) · Google
Windows, macOS, LinuxGraphics hardware: NVIDIA, AMD, Apple silicon, Intel, Processor only
Check before you start
llama.cpp is open-source software under the MIT License, and Google publishes the Gemma 4 26B A4B QAT GGUF under Apache 2.0 on Hugging Face without gating.
Not verified
USASI has not tested this configuration.
Google's figure covers the model weights only; memory for the image projector and for the context size llama-server chooses is not documented.
Google notes its memory numbers may change with the inference tool and environment.
Google's table does not name this file; the GGUF in Google's repository is 14,439,363,584 bytes on its own (Hugging Face metadata), about the same as the table's approximate 14.4 GB.
Which pre-built llama.cpp packages include AMD or Intel GPU support is not established here.
These figures are recorded as memory or system memory (RAM), not graphics memory, so the finder compares them with system memory and never with VRAM. They are not counted above. Enter your system memory in the finder to compare.
Every configuration in the finder records its publisher's own memory statement for one exact variant. A configuration is listed here when the finder, given 16 GB of graphics memory and “on my computer”, finds that every answer is met and that the publisher's figure is at or below 16 GB. It is the same rule the finder uses, not a separate estimate. Only configurations whose documentation lists NVIDIA, AMD or Intel graphics are considered, a graphics-memory figure is compared only with graphics memory, and a figure stated as system memory is never counted against VRAM.
What this does not tell you
Speed: how quickly a model responds on your hardware is not documented here.
Context length: a publisher's figure can leave out memory for the context window, which grows with longer prompts and responses, and for other software running at the same time. Read each publisher's statement.
Operating system, drivers, and graphics support: check each entry's documented platforms and graphics hardware.
Testing: USASI has not run these configurations unless an entry says it was tested.