Independent project. Not a U.S. government website.

Support our project (opens Buy Me a Coffee)
USASI

Jump to

Start hereFour ways into the catalogCompanies & LabsU.S. AI organizationsOpen Models & ToolsOpen models, software, datasets, and evaluation toolsLearnExplainers and learning pathsFor kidsAI and Super Intelligence, explained for readers 8 to 12Latest newsSourced news itemsPolicy trackerFederal AI laws, orders, and standards

Local AI by memory

AI models you can run on an 8 GB Apple silicon Mac

With 8 GB of unified memory on an Apple silicon Mac, you can run 6 documented local configurations in this catalog by their publishers' own memory figures, including llama.cpp with Llama 3.1 8B Instruct quantized to Q4_K_M (about 4.9 GB), LM Studio with Olmo 3 7B Instruct (about 6 GB), and LM Studio with Rnj-1 Instruct (about 6 GB). These are estimates from each publisher's documentation; USASI has not tested them.

Open the finder with 8 GB Apple siliconChoose another amount

6 configurations fit 8 GB of Apple silicon unified memory

Ordered by the publisher's memory figure, smallest first. This is not a ranking. Each entry names the exact variant the figure is for.

Estimated to fit, not testedOn your computerDeveloper setup

llama.cpp with Llama 3.1 8B Instruct quantized to Q4_K_M

llama.cpp includes a script to convert a Hugging Face model to GGUF, a llama-quantize tool to shrink it, and llama-server to run it with a web chat interface. Meta's Llama 3.1 8B Instruct is the instruction-tuned 8B model in the Llama 3.1 collection, which the model card says is intended for assistant-like chat.

Model
Llama 3.1 8B Instruct (meta-llama/Llama-3.1-8B-Instruct)
Variant
Llama 3.1 8B Instruct quantized to Q4_K_M with llama-quantize (llama-3.1-8b-instruct-Q4_K_M.gguf)
Documented memory
About 4.9 GB of memoryPublisher's statement: llama.cpp's quantize README gives no memory figure of its own for running this file. Its example table lists 4.9 GB as the quantized size (Q4_K_M) of Llama 3.1 8B, a file size, against an original size of 32.1 GB, and it says that because models are currently fully loaded into memory, memory and disk requirements are the same.Source: llama.cpp quantize README (external site: raw.githubusercontent.com) · ggml-org (GitHub)
Runtime and tools
Runtime: llama.cpp (convert_hf_to_gguf.py, llama-quantize, llama-server)
License
Llama 3.1 Community License Agreement (weights)As recorded, with sources, on the model's catalog record.
Documented for
Windows, macOS, LinuxGraphics hardware: NVIDIA, AMD, Apple silicon, Intel, Processor only

Check before you start

  • llama.cpp is MIT-licensed, and Meta grants a royalty-free license to Llama 3.1 under its Llama 3.1 Community License Agreement, a custom license with an acceptable use policy and additional commercial terms for very large services.
  • Needs an online account.

Not verified

  • USASI has not tested this configuration.
  • The 4.9 GB is a file size for Llama 3.1 8B in general, not a measured memory use; the README's detailed table (4.58 GiB at Q4_K_M) was measured on the pretrained Llama-3.1-8B, not the Instruct model.
  • The README does not say how much memory the context window adds on top of the model file.
  • Which pre-built llama.cpp packages include AMD or Intel GPU support is not established here.

Catalog record: Llama 3.1 8BOpen in the finder with 8 GB Apple silicon

Estimated to fit, not testedOn your computerInstaller app

LM Studio with Olmo 3 7B Instruct

LM Studio is a desktop app for downloading and running language models on your computer, with a chat interface and document attachments. Its model catalog includes Ai2's Olmo 3 7B as allenai/olmo-3-7b, built from Olmo-3-7B-Instruct, a chat model that Ai2 releases under Apache 2.0.

Model
Olmo 3 7B Instruct (LM Studio catalog entry allenai/olmo-3-7b)
Variant
LM Studio's allenai/olmo-3-7b entry (lmstudio-community Olmo-3-7B-Instruct GGUF or MLX)
Documented memory
About 6 GB of memoryPublisher's statement: LM Studio's catalog page for allenai/olmo-3-7b lists a minimum system memory of 6 GB. The entry is based on lmstudio-community's Olmo-3-7B-Instruct GGUF, offered as Q4_K_M, Q6_K, and Q8_0 files, and on 4-bit, 6-bit, and 8-bit MLX builds.Sources: allenai/olmo-3-7b (LM Studio model catalog) (external site: lmstudio.ai) · LM Studio (Element Labs); lmstudio-community/Olmo-3-7B-Instruct-GGUF repository metadata and file sizes (Hugging Face API) (external site: huggingface.co) · Hugging Face
Runtime and tools
App: LM Studio; Runtime: llama.cpp engine inside LM Studio; Runtime: MLX engine inside LM Studio (Apple silicon Macs)
License
Apache-2.0 (weights); Apache-2.0 (code); Apache-2.0 (code)As recorded, with sources, on the model's catalog record.
Documented for
Windows, macOS, LinuxGraphics hardware: Apple silicon, Processor only

Check before you start

  • LM Studio's Free plan includes running local models with llama.cpp and MLX, and Ai2 licenses Olmo 3 7B Instruct under Apache 2.0. Paid LM Studio plans add hosted open models and more included usage.

Not verified

  • USASI has not tested this configuration.
  • LM Studio's page does not say which download option its 6 GB minimum refers to; the Q8_0 GGUF alone is 7,759,251,776 bytes (Hugging Face file listing), so the minimum cannot cover that file.
  • LM Studio's pages reviewed do not name supported NVIDIA, AMD, or Intel graphics on Windows or Linux, so GPU acceleration on those cards is not established here.

Catalog record: Olmo 3 7B InstructOpen in the finder with 8 GB Apple silicon

Estimated to fit, not testedOn your computerInstaller app

LM Studio with Rnj-1 Instruct

LM Studio is a desktop app for downloading and running language models on your computer, with a chat interface and a local OpenAI-compatible server. Its model catalog includes Essential AI's Rnj-1 as essentialai/rnj-1, built from rnj-1-instruct, an 8B model that Essential AI describes as optimized for code and STEM tasks, including tool calling and agentic coding.

Model
Rnj-1 Instruct (LM Studio catalog entry essentialai/rnj-1)
Variant
LM Studio's essentialai/rnj-1 entry (lmstudio-community rnj-1-instruct GGUF)
Documented memory
About 6 GB of memoryPublisher's statement: LM Studio's catalog page for essentialai/rnj-1 lists a minimum system memory of 6 GB. The entry is based on lmstudio-community's rnj-1-instruct GGUF, offered as Q4_K_M, Q6_K, and Q8_0 files.Sources: essentialai/rnj-1 (LM Studio model catalog) (external site: lmstudio.ai) · LM Studio (Element Labs); lmstudio-community/rnj-1-instruct-GGUF repository metadata and file sizes (Hugging Face API) (external site: huggingface.co) · Hugging Face
Runtime and tools
App: LM Studio; Runtime: llama.cpp engine inside LM Studio
License
Apache-2.0 (weights)As recorded, with sources, on the model's catalog record.
Documented for
Windows, macOS, LinuxGraphics hardware: Apple silicon, Processor only

Check before you start

  • LM Studio's Free plan includes running local models with llama.cpp and MLX, and Essential AI licenses rnj-1-instruct under Apache 2.0. Paid LM Studio plans add hosted open models and more included usage.

Not verified

  • USASI has not tested this configuration.
  • LM Studio's page does not say which download option its 6 GB minimum refers to; the Q6_K and Q8_0 files alone are 6,826,763,264 and 8,839,332,864 bytes (Hugging Face file listing), so the minimum cannot cover them.
  • How well Rnj-1 works with a coding tool connected to LM Studio's local server is not documented.
  • LM Studio's pages reviewed do not name supported NVIDIA, AMD, or Intel graphics on Windows or Linux, so GPU acceleration on those cards is not established here.

Catalog record: Rnj-1 InstructOpen in the finder with 8 GB Apple silicon

Estimated to fit, not testedOn your computerCommand line or extension

OpenAI Whisper turbo on your own computer

Whisper is OpenAI's open-source speech recognition model, released with a Python package and a command-line tool under the MIT License. Its default turbo model, an optimized version of large-v3, transcribes audio files on your own computer.

Model
Whisper turbo (large-v3-turbo checkpoint)
Variant
Whisper turbo (large-v3-turbo checkpoint) in the openai-whisper package
Documented memory
About 6 GB of graphics memoryPublisher's statement: The Whisper README lists approximate memory requirements per model size, giving about 6 GB of required VRAM for the turbo model; the package maps the name turbo to the large-v3-turbo checkpoint.Sources: Whisper README (external site: raw.githubusercontent.com) · OpenAI (GitHub); whisper/__init__.py (model download table) (external site: raw.githubusercontent.com) · OpenAI (GitHub)
Runtime and tools
Library: openai-whisper (Python package and command line)
License
MIT (weights and code)As recorded, with sources, on the model's catalog record.
Documented for
Windows, macOS, LinuxGraphics hardware: NVIDIA, Processor only

Check before you start

  • Apple silicon graphics are not listed; it is documented to run on the processor, which may be slower.
  • Whisper's code and model weights are released under the MIT License and installed with pip; the model weights download from public links built into the package.

Not verified

  • USASI has not tested this configuration.
  • Speed on your hardware is not documented; the README's relative speeds were measured on an A100 GPU.
  • Acceleration on AMD, Intel, or Apple GPUs is not documented in the Whisper repository.

Catalog record: Whisper large-v3-turboOpen in the finder with 8 GB Apple silicon

Estimated to fit, not testedOn your computerCommand line or extension

llama.cpp server with Gemma 4 12B (QAT Q4_0 GGUF)

llama.cpp's llama-server runs a GGUF model on your own computer and serves a built-in web chat interface and an OpenAI-compatible API. Google publishes Gemma 4 12B as a Q4_0 GGUF built with quantization-aware training and names this download format for local use in llama.cpp.

Model
Gemma 4 12B instruction-tuned, QAT Q4_0 GGUF
Variant
Gemma 4 12B at Q4_0 (4-bit), as in Google's gemma-4-12B-it-qat-q4_0-gguf
Documented memory
About 6.7 GB of graphics memoryPublisher's statement: Google's Gemma 4 documentation lists about 6.7 GB of GPU or TPU memory to load Gemma 4 12B at Q4_0 (4-bit), based on the model weights plus a 20% loading overhead. It says the figure does not include memory for supporting software or the context window, which grows with the length of prompts and responses.Source: Gemma 4 overview (Gemma documentation) (external site: ai.google.dev) · Google
Runtime and tools
Runtime: llama.cpp (llama-server and its web UI)
License
Apache-2.0 (weights)As recorded, with sources, on the model's catalog record.
Documented for
Windows, macOS, LinuxGraphics hardware: NVIDIA, AMD, Apple silicon, Intel, Processor only

Check before you start

  • llama.cpp is open-source software under the MIT License, and Google publishes the Gemma 4 12B QAT GGUF under Apache 2.0 on Hugging Face without gating.

Not verified

  • USASI has not tested this configuration.
  • Google's memory figure excludes the context window; total memory at the context size llama-server chooses on your hardware is not documented.
  • Google's repository also holds an mmproj file for 12B, but llama.cpp's multimodal page lists Gemma 4 E2B, E4B, 26B A4B, and 31B for image input and not 12B, so image input is not established here.
  • Google's table does not name this file; the GGUF in Google's repository is 6,975,879,296 bytes on its own (Hugging Face metadata), slightly more than the table's approximate 6.7 GB.
  • Which pre-built llama.cpp packages include AMD or Intel GPU support is not established here.

Catalog record: Gemma 4 12B UnifiedOpen in the finder with 8 GB Apple silicon

Estimated to fit, not testedOn your computerInstaller app

LM Studio with Gemma 4 12B (QAT Q4_0 GGUF)

LM Studio is a desktop app for downloading and running language models on your computer, with a chat interface and document attachments. Google's Gemma 4 documentation names a Q4_0 GGUF download, built with quantization-aware training, for local use in llama.cpp and LM Studio.

Model
Gemma 4 12B instruction-tuned, QAT Q4_0 GGUF
Variant
Gemma 4 12B at Q4_0 (4-bit), such as Google's gemma-4-12B-it-qat-q4_0-gguf
Documented memory
About 6.7 GB of graphics memoryPublisher's statement: Google's Gemma 4 documentation lists about 6.7 GB of GPU or TPU memory to load Gemma 4 12B at Q4_0 (4-bit), based on the model weights plus a 20% loading overhead. It says the figure does not include memory for supporting software or the context window, which grows with the length of prompts and responses.Sources: Gemma 4 overview (Gemma documentation) (external site: ai.google.dev) · Google; gemma-4-12B-it-qat-q4_0-gguf model card (external site: huggingface.co) · Google (Hugging Face)
Runtime and tools
App: LM Studio; Runtime: llama.cpp engine inside LM Studio
License
Apache-2.0 (weights)As recorded, with sources, on the model's catalog record.
Documented for
Windows, macOS, LinuxGraphics hardware: Apple silicon, Processor only

Check before you start

  • LM Studio's Free plan includes running local models with llama.cpp and MLX, and the Gemma 4 weights are Apache 2.0. Paid LM Studio plans add hosted cloud models and more included usage.

Not verified

  • USASI has not tested this configuration.
  • LM Studio's pages reviewed do not name supported NVIDIA, AMD, or Intel graphics on Windows or Linux, so GPU acceleration on those cards is not established here.
  • Google's memory figure excludes the context window; total memory at the context length you choose is not documented.

Catalog record: Gemma 4 12B UnifiedOpen in the finder with 8 GB Apple silicon

Documented, but needs more than 8 GB (8)

These would otherwise match, but the publisher's figure is higher than this amount.

Runs locally, but no memory figure is published (4)

The publisher does not state a memory figure for a specific variant, so USASI does not estimate one and these are not counted above.

How this list is made

Every configuration in the finder records its publisher's own memory statement for one exact variant. A configuration is listed here when the finder, given macOS on Apple silicon with 8 GB of memory and “on my computer”, finds that every answer is met and that the publisher's figure is at or below 8 GB. It is the same rule the finder uses, not a separate estimate. Apple silicon shares one pool of unified memory, so a graphics-memory figure and a system-memory figure are both compared with it.

What this does not tell you

  • Speed: how quickly a model responds on your hardware is not documented here.
  • Context length: a publisher's figure can leave out memory for the context window, which grows with longer prompts and responses, and for other software running at the same time. Read each publisher's statement.
  • Operating system, drivers, and graphics support: check each entry's documented platforms and graphics hardware.
  • Testing: USASI has not run these configurations unless an entry says it was tested.

To estimate memory for a model and precision yourself, use the model size calculator, or read understanding inference hardware.

Choose another amount

The number in brackets is how many documented configurations fit each amount.

All amounts · Running AI on your own hardware · Find AI for my needs

Support Us

Help keep USASI useful.

Leave a tip on Buy Me a Coffee (external site)

Optional. No USASI account required. Payment takes place on the linked provider’s website (Buy Me a Coffee).

About supporting this project