Local AI by memory

AI models you can run on a 32 GB Apple silicon Mac

With 32 GB of unified memory on an Apple silicon Mac, you can run 14 documented local configurations in this catalog by their publishers' own memory figures, including llama.cpp with Llama 3.1 8B Instruct quantized to Q4_K_M (about 4.9 GB), LM Studio with Olmo 3 7B Instruct (about 6 GB), and LM Studio with Rnj-1 Instruct (about 6 GB). These are estimates from each publisher's documentation; USASI has not tested them.

Open the finder with 32 GB Apple silicon Choose another amount

14 configurations fit 32 GB of Apple silicon unified memory

Ordered by the publisher's memory figure, smallest first. This is not a ranking. Each entry names the exact variant the figure is for.

Estimated to fit, not testedOn your computerDeveloper setup

llama.cpp with Llama 3.1 8B Instruct quantized to Q4_K_M

llama.cpp includes a script to convert a Hugging Face model to GGUF, a llama-quantize tool to shrink it, and llama-server to run it with a web chat interface. Meta's Llama 3.1 8B Instruct is the instruction-tuned 8B model in the Llama 3.1 collection, which the model card says is intended for assistant-like chat.

Model: Llama 3.1 8B Instruct (meta-llama/Llama-3.1-8B-Instruct)
Variant: Llama 3.1 8B Instruct quantized to Q4_K_M with llama-quantize (llama-3.1-8b-instruct-Q4_K_M.gguf)
Documented memory: About 4.9 GB of memoryPublisher's statement: llama.cpp's quantize README gives no memory figure of its own for running this file. Its example table lists 4.9 GB as the quantized size (Q4_K_M) of Llama 3.1 8B, a file size, against an original size of 32.1 GB, and it says that because models are currently fully loaded into memory, memory and disk requirements are the same.Source: llama.cpp quantize README (external site: raw.githubusercontent.com) · ggml-org (GitHub)
Runtime and tools: Runtime: llama.cpp (convert_hf_to_gguf.py, llama-quantize, llama-server)
License: Llama 3.1 Community License Agreement (weights)As recorded, with sources, on the model's catalog record.
Documented for: Windows, macOS, LinuxGraphics hardware: NVIDIA, AMD, Apple silicon, Intel, Processor only

Check before you start

llama.cpp is MIT-licensed, and Meta grants a royalty-free license to Llama 3.1 under its Llama 3.1 Community License Agreement, a custom license with an acceptable use policy and additional commercial terms for very large services.
Needs an online account.

Not verified

USASI has not tested this configuration.
The 4.9 GB is a file size for Llama 3.1 8B in general, not a measured memory use; the README's detailed table (4.58 GiB at Q4_K_M) was measured on the pretrained Llama-3.1-8B, not the Instruct model.
The README does not say how much memory the context window adds on top of the model file.
Which pre-built llama.cpp packages include AMD or Intel GPU support is not established here.

Catalog record: Llama 3.1 8B Open in the finder with 32 GB Apple silicon

Estimated to fit, not testedOn your computerInstaller app

LM Studio with Olmo 3 7B Instruct

LM Studio is a desktop app for downloading and running language models on your computer, with a chat interface and document attachments. Its model catalog includes Ai2's Olmo 3 7B as allenai/olmo-3-7b, built from Olmo-3-7B-Instruct, a chat model that Ai2 releases under Apache 2.0.

Model: Olmo 3 7B Instruct (LM Studio catalog entry allenai/olmo-3-7b)
Variant: LM Studio's allenai/olmo-3-7b entry (lmstudio-community Olmo-3-7B-Instruct GGUF or MLX)
Documented memory: About 6 GB of memoryPublisher's statement: LM Studio's catalog page for allenai/olmo-3-7b lists a minimum system memory of 6 GB. The entry is based on lmstudio-community's Olmo-3-7B-Instruct GGUF, offered as Q4_K_M, Q6_K, and Q8_0 files, and on 4-bit, 6-bit, and 8-bit MLX builds.Sources: allenai/olmo-3-7b (LM Studio model catalog) (external site: lmstudio.ai) · LM Studio (Element Labs); lmstudio-community/Olmo-3-7B-Instruct-GGUF repository metadata and file sizes (Hugging Face API) (external site: huggingface.co) · Hugging Face
Runtime and tools: App: LM Studio; Runtime: llama.cpp engine inside LM Studio; Runtime: MLX engine inside LM Studio (Apple silicon Macs)
License: Apache-2.0 (weights); Apache-2.0 (code); Apache-2.0 (code)As recorded, with sources, on the model's catalog record.
Documented for: Windows, macOS, LinuxGraphics hardware: Apple silicon, Processor only

Check before you start

LM Studio's Free plan includes running local models with llama.cpp and MLX, and Ai2 licenses Olmo 3 7B Instruct under Apache 2.0. Paid LM Studio plans add hosted open models and more included usage.

Not verified

USASI has not tested this configuration.
LM Studio's page does not say which download option its 6 GB minimum refers to; the Q8_0 GGUF alone is 7,759,251,776 bytes (Hugging Face file listing), so the minimum cannot cover that file.
LM Studio's pages reviewed do not name supported NVIDIA, AMD, or Intel graphics on Windows or Linux, so GPU acceleration on those cards is not established here.

Catalog record: Olmo 3 7B Instruct Open in the finder with 32 GB Apple silicon

Estimated to fit, not testedOn your computerInstaller app

LM Studio with Rnj-1 Instruct

LM Studio is a desktop app for downloading and running language models on your computer, with a chat interface and a local OpenAI-compatible server. Its model catalog includes Essential AI's Rnj-1 as essentialai/rnj-1, built from rnj-1-instruct, an 8B model that Essential AI describes as optimized for code and STEM tasks, including tool calling and agentic coding.

Model: Rnj-1 Instruct (LM Studio catalog entry essentialai/rnj-1)
Variant: LM Studio's essentialai/rnj-1 entry (lmstudio-community rnj-1-instruct GGUF)
Documented memory: About 6 GB of memoryPublisher's statement: LM Studio's catalog page for essentialai/rnj-1 lists a minimum system memory of 6 GB. The entry is based on lmstudio-community's rnj-1-instruct GGUF, offered as Q4_K_M, Q6_K, and Q8_0 files.Sources: essentialai/rnj-1 (LM Studio model catalog) (external site: lmstudio.ai) · LM Studio (Element Labs); lmstudio-community/rnj-1-instruct-GGUF repository metadata and file sizes (Hugging Face API) (external site: huggingface.co) · Hugging Face
Runtime and tools: App: LM Studio; Runtime: llama.cpp engine inside LM Studio
License: Apache-2.0 (weights)As recorded, with sources, on the model's catalog record.
Documented for: Windows, macOS, LinuxGraphics hardware: Apple silicon, Processor only

Check before you start

LM Studio's Free plan includes running local models with llama.cpp and MLX, and Essential AI licenses rnj-1-instruct under Apache 2.0. Paid LM Studio plans add hosted open models and more included usage.

Not verified

USASI has not tested this configuration.
LM Studio's page does not say which download option its 6 GB minimum refers to; the Q6_K and Q8_0 files alone are 6,826,763,264 and 8,839,332,864 bytes (Hugging Face file listing), so the minimum cannot cover them.
How well Rnj-1 works with a coding tool connected to LM Studio's local server is not documented.
LM Studio's pages reviewed do not name supported NVIDIA, AMD, or Intel graphics on Windows or Linux, so GPU acceleration on those cards is not established here.

Catalog record: Rnj-1 Instruct Open in the finder with 32 GB Apple silicon

Estimated to fit, not testedOn your computerCommand line or extension

OpenAI Whisper turbo on your own computer

Whisper is OpenAI's open-source speech recognition model, released with a Python package and a command-line tool under the MIT License. Its default turbo model, an optimized version of large-v3, transcribes audio files on your own computer.

Model: Whisper turbo (large-v3-turbo checkpoint)
Variant: Whisper turbo (large-v3-turbo checkpoint) in the openai-whisper package
Documented memory: About 6 GB of graphics memoryPublisher's statement: The Whisper README lists approximate memory requirements per model size, giving about 6 GB of required VRAM for the turbo model; the package maps the name turbo to the large-v3-turbo checkpoint.Sources: Whisper README (external site: raw.githubusercontent.com) · OpenAI (GitHub); whisper/__init__.py (model download table) (external site: raw.githubusercontent.com) · OpenAI (GitHub)
Runtime and tools: Library: openai-whisper (Python package and command line)
License: MIT (weights and code)As recorded, with sources, on the model's catalog record.
Documented for: Windows, macOS, LinuxGraphics hardware: NVIDIA, Processor only

Check before you start

Apple silicon graphics are not listed; it is documented to run on the processor, which may be slower.
Whisper's code and model weights are released under the MIT License and installed with pip; the model weights download from public links built into the package.

Not verified

USASI has not tested this configuration.
Speed on your hardware is not documented; the README's relative speeds were measured on an A100 GPU.
Acceleration on AMD, Intel, or Apple GPUs is not documented in the Whisper repository.

Catalog record: Whisper large-v3-turbo Open in the finder with 32 GB Apple silicon

Estimated to fit, not testedOn your computerCommand line or extension

llama.cpp server with Gemma 4 12B (QAT Q4_0 GGUF)

llama.cpp's llama-server runs a GGUF model on your own computer and serves a built-in web chat interface and an OpenAI-compatible API. Google publishes Gemma 4 12B as a Q4_0 GGUF built with quantization-aware training and names this download format for local use in llama.cpp.

Model: Gemma 4 12B instruction-tuned, QAT Q4_0 GGUF
Variant: Gemma 4 12B at Q4_0 (4-bit), as in Google's gemma-4-12B-it-qat-q4_0-gguf
Documented memory: About 6.7 GB of graphics memoryPublisher's statement: Google's Gemma 4 documentation lists about 6.7 GB of GPU or TPU memory to load Gemma 4 12B at Q4_0 (4-bit), based on the model weights plus a 20% loading overhead. It says the figure does not include memory for supporting software or the context window, which grows with the length of prompts and responses.Source: Gemma 4 overview (Gemma documentation) (external site: ai.google.dev) · Google
Runtime and tools: Runtime: llama.cpp (llama-server and its web UI)
License: Apache-2.0 (weights)As recorded, with sources, on the model's catalog record.
Documented for: Windows, macOS, LinuxGraphics hardware: NVIDIA, AMD, Apple silicon, Intel, Processor only

Check before you start

llama.cpp is open-source software under the MIT License, and Google publishes the Gemma 4 12B QAT GGUF under Apache 2.0 on Hugging Face without gating.

Not verified

USASI has not tested this configuration.
Google's memory figure excludes the context window; total memory at the context size llama-server chooses on your hardware is not documented.
Google's repository also holds an mmproj file for 12B, but llama.cpp's multimodal page lists Gemma 4 E2B, E4B, 26B A4B, and 31B for image input and not 12B, so image input is not established here.
Google's table does not name this file; the GGUF in Google's repository is 6,975,879,296 bytes on its own (Hugging Face metadata), slightly more than the table's approximate 6.7 GB.
Which pre-built llama.cpp packages include AMD or Intel GPU support is not established here.

Catalog record: Gemma 4 12B Unified Open in the finder with 32 GB Apple silicon

Estimated to fit, not testedOn your computerInstaller app

LM Studio with Gemma 4 12B (QAT Q4_0 GGUF)

LM Studio is a desktop app for downloading and running language models on your computer, with a chat interface and document attachments. Google's Gemma 4 documentation names a Q4_0 GGUF download, built with quantization-aware training, for local use in llama.cpp and LM Studio.

Model: Gemma 4 12B instruction-tuned, QAT Q4_0 GGUF
Variant: Gemma 4 12B at Q4_0 (4-bit), such as Google's gemma-4-12B-it-qat-q4_0-gguf
Documented memory: About 6.7 GB of graphics memoryPublisher's statement: Google's Gemma 4 documentation lists about 6.7 GB of GPU or TPU memory to load Gemma 4 12B at Q4_0 (4-bit), based on the model weights plus a 20% loading overhead. It says the figure does not include memory for supporting software or the context window, which grows with the length of prompts and responses.Sources: Gemma 4 overview (Gemma documentation) (external site: ai.google.dev) · Google; gemma-4-12B-it-qat-q4_0-gguf model card (external site: huggingface.co) · Google (Hugging Face)
Runtime and tools: App: LM Studio; Runtime: llama.cpp engine inside LM Studio
License: Apache-2.0 (weights)As recorded, with sources, on the model's catalog record.
Documented for: Windows, macOS, LinuxGraphics hardware: Apple silicon, Processor only

Check before you start

LM Studio's Free plan includes running local models with llama.cpp and MLX, and the Gemma 4 weights are Apache 2.0. Paid LM Studio plans add hosted cloud models and more included usage.

Not verified

USASI has not tested this configuration.
LM Studio's pages reviewed do not name supported NVIDIA, AMD, or Intel graphics on Windows or Linux, so GPU acceleration on those cards is not established here.
Google's memory figure excludes the context window; total memory at the context length you choose is not documented.

Catalog record: Gemma 4 12B Unified Open in the finder with 32 GB Apple silicon

Estimated to fit, not testedOn your computerCommand line or extension

OpenAI Whisper large-v3 on your own computer

Whisper is OpenAI's open-source speech recognition model, released with a Python package and a command-line tool under the MIT License. Its large model, which the package maps to the large-v3 checkpoint, transcribes speech and translates non-English speech into English on your own computer.

Model: Whisper large (large-v3 checkpoint)
Variant: Whisper large (large-v3 checkpoint) in the openai-whisper package
Documented memory: About 10 GB of graphics memoryPublisher's statement: The Whisper README lists approximate memory requirements per model size, giving about 10 GB of required VRAM for the large model; the package maps the name large to the large-v3 checkpoint.Sources: Whisper README (external site: raw.githubusercontent.com) · OpenAI (GitHub); whisper/__init__.py (model download table and device selection) (external site: raw.githubusercontent.com) · OpenAI (GitHub)
Runtime and tools: Library: openai-whisper (Python package and command line)
License: MIT (weights and code); Apache-2.0 (weights)As recorded, with sources, on the model's catalog record.
Documented for: Windows, macOS, LinuxGraphics hardware: NVIDIA, Processor only

Check before you start

Apple silicon graphics are not listed; it is documented to run on the processor, which may be slower.
Whisper's code and model weights are released under the MIT License and installed with pip; the model weights download from public links built into the package.

Not verified

USASI has not tested this configuration.
Speed on your hardware is not documented; the README's relative speeds were measured on an A100 GPU.
Acceleration on AMD, Intel, or Apple GPUs is not documented in the Whisper repository.
The ~10 GB figure is stated as VRAM; on a Mac the package picks the CPU (it checks only for CUDA) and uses FP32, and memory use there is not documented.

Catalog record: Whisper large-v3 Open in the finder with 32 GB Apple silicon

Estimated to fit, not testedOn your computerCommand line or extension

llama.cpp server with Gemma 4 26B A4B (QAT Q4_0 GGUF)

llama.cpp's llama-server runs a GGUF model on your own computer and serves a built-in web chat interface and an OpenAI-compatible API. Google publishes Gemma 4 26B A4B, a mixture-of-experts model with about 3.8 billion of its 25.2 billion parameters active per token, as a Q4_0 GGUF built with quantization-aware training, together with a projector file for image input.

Model: Gemma 4 26B A4B instruction-tuned, QAT Q4_0 GGUF
Variant: Gemma 4 26B A4B at Q4_0 (4-bit), as in Google's gemma-4-26B-A4B-it-qat-q4_0-gguf
Documented memory: About 14.4 GB of graphics memoryPublisher's statement: Google's Gemma 4 documentation lists about 14.4 GB of GPU or TPU memory to load Gemma 4 26B A4B at Q4_0 (4-bit), based on the model weights plus a 20% loading overhead. It explains that all 26 billion parameters must be loaded even though only about 4 billion are active per token, and that the figure excludes supporting software and the context window.Source: Gemma 4 overview (Gemma documentation) (external site: ai.google.dev) · Google
Runtime and tools: Runtime: llama.cpp (llama-server and its web UI)
License: Apache-2.0 (weights)As recorded, with sources, on the model's catalog record.
Documented for: Windows, macOS, LinuxGraphics hardware: NVIDIA, AMD, Apple silicon, Intel, Processor only

Check before you start

llama.cpp is open-source software under the MIT License, and Google publishes the Gemma 4 26B A4B QAT GGUF under Apache 2.0 on Hugging Face without gating.

Not verified

USASI has not tested this configuration.
Google's figure covers the model weights only; memory for the image projector and for the context size llama-server chooses is not documented.
Google notes its memory numbers may change with the inference tool and environment.
Google's table does not name this file; the GGUF in Google's repository is 14,439,363,584 bytes on its own (Hugging Face metadata), about the same as the table's approximate 14.4 GB.
Which pre-built llama.cpp packages include AMD or Intel GPU support is not established here.

Catalog record: Gemma 4 26B A4B Open in the finder with 32 GB Apple silicon

Estimated to fit, not testedOn your computerCommand line or extension

Cline in VS Code with gpt-oss-20b on Ollama

Cline is an open-source coding agent that runs as an editor extension and can use a model served by Ollama on your own computer. OpenAI describes gpt-oss-20b, one of its open-weight models, as meant for lower latency and local or specialized use cases, and Ollama offers it as gpt-oss:20b.

Model: gpt-oss-20b (Ollama tag gpt-oss:20b)
Variant: gpt-oss-20b with MXFP4-quantized MoE weights (Ollama tag gpt-oss:20b)
Documented memory: About 16 GB of memoryPublisher's statement: OpenAI's model card says MXFP4 quantization of the MoE weights lets gpt-oss-20b "run within 16GB of memory". Ollama's library page says it supports MXFP4 natively without additional quantization and that the smaller model can run on systems with as little as 16GB memory.Sources: gpt-oss-20b model card (external site: huggingface.co) · OpenAI (Hugging Face); gpt-oss in the Ollama library (external site: ollama.com) · Ollama
Runtime and tools: Extension: Cline; App: Visual Studio Code (external site: code.visualstudio.com); Runtime: Ollama
License: Apache-2.0 (weights); Apache-2.0 (code)As recorded, with sources, on the model's catalog record.
Documented for: Windows, macOS, LinuxGraphics hardware: NVIDIA, AMD, Apple silicon, Intel, Processor only

Check before you start

Cline and the gpt-oss-20b weights are Apache 2.0, and Ollama's pricing page says running models on your own hardware is always unlimited. Cline's hosted provider and Ollama's cloud models are separate options.

Not verified

USASI has not tested this configuration.
The 16 GB figure is the publishers' statement for the model; how much a longer context adds on your hardware is not documented.
Neither publisher documents how well gpt-oss-20b handles Cline's agent tasks.
Ollama names Intel GPUs only in its Linux Vulkan driver instructions and lists no supported Intel models; Intel GPU acceleration on Windows is not established here, and Intel-based Macs run on the CPU only.

Catalog record: gpt-oss-20b Open in the finder with 32 GB Apple silicon

Estimated to fit, not testedOn your computerInstaller app

LM Studio with gpt-oss-20b

LM Studio is a desktop app for downloading and running language models on your computer, with a chat interface and document attachments. OpenAI's guide to running its open-weight gpt-oss-20b in LM Studio covers downloading the model as openai/gpt-oss-20b, chatting with it, and connecting MCP servers as tools.

Model: gpt-oss-20b (LM Studio model openai/gpt-oss-20b)
Variant: LM Studio's openai/gpt-oss-20b entry (lmstudio-community gpt-oss-20b-MXFP4.gguf, or mlx-community MXFP4-Q8)
Documented memory: About 16 GB of graphics memoryPublisher's statement: OpenAI's guide to running gpt-oss with LM Studio says gpt-oss-20b only requires at least 16GB of VRAM, and OpenAI's model card says MXFP4 quantization of the MoE weights lets it run within 16GB of memory. LM Studio's catalog page for openai/gpt-oss-20b, based on lmstudio-community/gpt-oss-20b-GGUF and mlx-community/gpt-oss-20b-MXFP4-Q8, lists a minimum system memory of 12 GB.Sources: How to run gpt-oss locally with LM Studio (OpenAI Cookbook) (external site: cookbook.openai.com) · OpenAI; gpt-oss-20b model card (external site: huggingface.co) · OpenAI (Hugging Face); openai/gpt-oss-20b (LM Studio model catalog) (external site: lmstudio.ai) · LM Studio (Element Labs)
Runtime and tools: App: LM Studio; Runtime: llama.cpp engine inside LM Studio
License: Apache-2.0 (weights); Apache-2.0 (code)As recorded, with sources, on the model's catalog record.
Documented for: Windows, macOS, LinuxGraphics hardware: Apple silicon, Processor only

Check before you start

LM Studio's Free plan includes running local models with llama.cpp and MLX, and OpenAI releases gpt-oss-20b under Apache 2.0. Paid LM Studio plans add hosted open models and more included usage.

Not verified

USASI has not tested this configuration.
The 16 GB figure is OpenAI's statement for the model; how much a longer context adds on your hardware is not documented.
LM Studio's pages reviewed do not name supported NVIDIA, AMD, or Intel graphics on Windows or Linux, so GPU acceleration on those cards is not established here.

Catalog record: gpt-oss-20b Open in the finder with 32 GB Apple silicon

Estimated to fit, not testedOn your computerInstaller app

Ollama desktop app with gpt-oss-20b

Ollama's desktop app for macOS and Windows lets you download models and chat with them, and you can drag text files or PDFs into a chat. OpenAI's open-weight gpt-oss-20b is available in it as gpt-oss:20b.

Model: gpt-oss-20b (Ollama tag gpt-oss:20b)
Variant: gpt-oss-20b with MXFP4-quantized MoE weights (Ollama tag gpt-oss:20b)
Documented memory: About 16 GB of memoryPublisher's statement: OpenAI's model card says MXFP4 quantization of the MoE weights lets gpt-oss-20b "run within 16GB of memory". Ollama's library page says it supports MXFP4 natively without additional quantization and that the smaller model can run on systems with as little as 16GB memory.Sources: gpt-oss-20b model card (external site: huggingface.co) · OpenAI (Hugging Face); gpt-oss in the Ollama library (external site: ollama.com) · Ollama
Runtime and tools: App: Ollama desktop app
License: Apache-2.0 (weights); Apache-2.0 (code)As recorded, with sources, on the model's catalog record.
Documented for: macOS, WindowsGraphics hardware: NVIDIA, AMD, Apple silicon, Intel, Processor only

Check before you start

Ollama's Free plan includes running models locally, and its pricing page says running models on your own hardware is always unlimited; gpt-oss-20b is Apache 2.0. Ollama's optional cloud models draw on included or purchased usage credits.

Not verified

USASI has not tested this configuration.
Ollama's app announcement mentions text and PDF drag and drop but does not document file size or page limits.
The 16 GB figure does not include memory for a longer context.
Ollama names Intel GPUs only in its Linux Vulkan driver instructions and lists no supported Intel models; Intel GPU acceleration on Windows is not established here, and Intel-based Macs run on the CPU only.

Catalog record: gpt-oss-20b Open in the finder with 32 GB Apple silicon

Estimated to fit, not testedOn your computerInstaller app

LM Studio with Gemma 4 31B (QAT Q4_0 GGUF)

Model: Gemma 4 31B instruction-tuned, QAT Q4_0 GGUF
Variant: Gemma 4 31B at Q4_0 (4-bit), as in Google's gemma-4-31B-it-qat-q4_0-gguf
Documented memory: About 17.5 GB of graphics memoryPublisher's statement: Google's Gemma 4 documentation lists about 17.5 GB of GPU or TPU memory to load Gemma 4 31B at Q4_0 (4-bit), based on the model weights plus a 20% loading overhead. It says the figure does not include memory for supporting software or the context window, which grows with the length of prompts and responses.Source: Gemma 4 overview (Gemma documentation) (external site: ai.google.dev) · Google
Runtime and tools: App: LM Studio; Runtime: llama.cpp engine inside LM Studio
License: Apache-2.0 (weights)As recorded, with sources, on the model's catalog record.
Documented for: Windows, macOS, LinuxGraphics hardware: Apple silicon, Processor only

Check before you start

LM Studio's Free plan includes running local models with llama.cpp and MLX, and Google publishes this Gemma 4 GGUF under Apache 2.0. Paid LM Studio plans add hosted open models and more included usage.

Not verified

USASI has not tested this configuration.
LM Studio's pages reviewed do not name supported NVIDIA, AMD, or Intel graphics on Windows or Linux, so GPU acceleration on those cards is not established here.
Google's memory figure excludes the context window; total memory at the context length you choose is not documented.
Whether LM Studio loads Google's separate image projector file for this GGUF is not documented in the pages reviewed.
Google's table does not name this file; the GGUF in Google's repository is 17,651,001,568 bytes on its own (Hugging Face metadata), slightly more than the table's approximate 17.5 GB.

Catalog record: Gemma 4 31B Open in the finder with 32 GB Apple silicon

Estimated to fit, not testedOn your computerCommand line or extension

llama.cpp server with Muse Glimmer 30B (KQuant-17GB Q4_K_M GGUF)

Muse Glimmer is Meta's open-weight 30B model built for agentic work on consumer hardware, including coding agents, tool use, and reading screenshots, charts, and documents. Meta publishes GGUF builds for llama.cpp, including a 4-bit KQuant-17GB text model and a separate perception-encoder file for image input, which llama-server serves with a web chat interface and an OpenAI-compatible API.

Model: Muse Glimmer 30B, KQuant-17GB Q4_K_M GGUF
Variant: Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf, Meta's 4-bit GGUF build
Documented memory: About 24 GB of graphics memoryPublisher's statement: Meta's GGUF model card says the KQuant-17GB build fits 24 GB VRAM, and the Muse Glimmer model card lists 24GB VRAM as the target hardware for that quantization. The GGUF card puts rough total memory, weights plus a working context, at about 17 GB for text only, 19 GB with the perception encoder, and 20 GB with the speculative-decoding drafter as well.Sources: Muse-Glimmer-30B-GGUF model card (external site: huggingface.co) · Meta (Hugging Face); Muse Glimmer model card (meta-models/Muse-Glimmer-30B) (external site: huggingface.co) · Meta (Hugging Face)
Runtime and tools: Runtime: llama.cpp (llama-server and its web UI)
License: Apache-2.0 (weights)As recorded, with sources, on the model's catalog record.
Documented for: Windows, macOS, LinuxGraphics hardware: NVIDIA, Apple silicon, Processor only

Check before you start

Meta releases Muse Glimmer's weights, including the GGUF builds, under Apache 2.0, and llama.cpp is open-source software under the MIT License.

Not verified

USASI has not tested this configuration.
Meta's rough memory totals assume a working context; memory at the long context in the card's server example (131,072 tokens over four slots) is not stated.
Meta's card documents CUDA, Apple Metal, and CPU-only builds; AMD and Intel GPUs are not documented for this model in the pages reviewed.
How well the model works with a particular coding tool through llama-server is not documented.

Catalog record: Muse Glimmer 30B Open in the finder with 32 GB Apple silicon

Estimated to fit, not testedOn your computerInstaller app

LM Studio with Muse Glimmer 30B

LM Studio is a desktop app for downloading and running language models on your computer, with a chat interface and document attachments. Its model catalog includes Meta's Muse Glimmer 30B as meta/muse-glimmer, listed with vision input, tool use, and reasoning, and built from a 4-bit KQuant-17GB GGUF.

Model: Muse Glimmer 30B (LM Studio catalog entry meta/muse-glimmer)
Variant: LM Studio's meta/muse-glimmer entry (lmstudio-community KQuant-17GB Q4_K_M GGUF)
Documented memory: About 26 GB of memoryPublisher's statement: LM Studio's catalog page for meta/muse-glimmer lists a minimum system memory of 26 GB. The entry is based on lmstudio-community's Muse-Glimmer-30B-GGUF repository, which holds the KQuant-17GB Q4_K_M model file and a perception-encoder file for image input.Sources: meta/muse-glimmer (LM Studio model catalog) (external site: lmstudio.ai) · LM Studio (Element Labs); lmstudio-community/Muse-Glimmer-30B-GGUF repository metadata (Hugging Face API) (external site: huggingface.co) · Hugging Face
Runtime and tools: App: LM Studio; Runtime: llama.cpp engine inside LM Studio
License: Apache-2.0 (weights)As recorded, with sources, on the model's catalog record.
Documented for: Windows, macOS, LinuxGraphics hardware: Apple silicon, Processor only

Check before you start

LM Studio's Free plan includes running local models with llama.cpp and MLX, and Meta releases Muse Glimmer's weights under Apache 2.0. Paid LM Studio plans add hosted open models and more included usage.

Not verified

USASI has not tested this configuration.
LM Studio's page does not say what context length its 26 GB minimum assumes or whether it includes the image encoder.
LM Studio's pages reviewed do not name supported NVIDIA, AMD, or Intel graphics on Windows or Linux, so GPU acceleration on those cards is not established here.

Catalog record: Muse Glimmer 30B Open in the finder with 32 GB Apple silicon

Runs locally, but no memory figure is published (4)

The publisher does not state a memory figure for a specific variant, so USASI does not estimate one and these are not counted above.

AnythingLLM Desktop with Gemma 4 12B on Ollama
Model: Gemma 4 12B (Ollama tag gemma4:12b). Open in the finder
goose desktop agent with Granite 4.2 8B on Ollama
Model: Granite 4.2 8B (Ollama tag granite4.2:8b). Open in the finder
llamafile with gpt-oss-20b (MXFP4)
Model: gpt-oss-20b (gpt-oss-20b-mxfp4.llamafile). Open in the finder
Open WebUI with Granite 4.2 8B on Ollama
Model: Granite 4.2 8B (Ollama tag granite4.2:8b). Open in the finder

How this list is made

Every configuration in the finder records its publisher's own memory statement for one exact variant. A configuration is listed here when the finder, given macOS on Apple silicon with 32 GB of memory and “on my computer”, finds that every answer is met and that the publisher's figure is at or below 32 GB. It is the same rule the finder uses, not a separate estimate. Apple silicon shares one pool of unified memory, so a graphics-memory figure and a system-memory figure are both compared with it.

What this does not tell you

Speed: how quickly a model responds on your hardware is not documented here.
Context length: a publisher's figure can leave out memory for the context window, which grows with longer prompts and responses, and for other software running at the same time. Read each publisher's statement.
Operating system, drivers, and graphics support: check each entry's documented platforms and graphics hardware.
Testing: USASI has not run these configurations unless an entry says it was tested.

To estimate memory for a model and precision yourself, use the model size calculator, or read understanding inference hardware.

Choose another amount

The number in brackets is how many documented configurations fit each amount.

All amounts · Running AI on your own hardware · Find AI for my needs

Support Us

Help keep USASI useful.

Leave a tip

Optional. No USASI account required. Payment takes place on the linked provider’s website (Buy Me a Coffee).

About supporting this project