OnnxStream
by vitoplantamura (personal account)
Lightweight inference library for ONNX files, written in C++. It can run Stable Diffusion XL 1.0 on a RPI Zero 2 (or in 298MB of RAM) but also Mistral 7B on desktops and servers. ARM, x86, WASM, RISC-V supported. Accelerated by XNNPACK. Python, C# and JS(WASM) bindings available. (description from the project’s GitHub page)
Repository facts
- Stars
- 2,088
- Forks
- 99
- Open issues and pull requests
- 67
- Primary language
- C++
- License
- Other license
- Owner
- vitoplantamura · Personal account
- Created
- Jul 14, 2023
- Last push
- Jun 18, 2026
Topics
Related projects
Projects that share this one’s GitHub topics.
Rank 4,571: microsoft/ai-dev-gallery
An open-source project for Windows developers to learn how to add AI with local models and APIs to Windows apps.
1,509 starsC#MITLast push Oct 17, 2026
Rank 8,360: timmyy123/LLM-Hub
Local LLM, image&video&music generator, run codex with local models on your phone
617 starsCOther licenseLast push Oct 18, 2026
Rank 8,460: ai-techsystems/deepC
vendor independent TinyML deep learning library, compiler and inference framework microcomputers and micro-controllers
606 starsC++Apache-2.0Last push Jul 22, 2025
Rank 1,803: xlite-dev/lite.ai.toolkit
A lite C++ AI toolkit: 100+ models with MNN, ORT and TRT, including Det, Seg, Stable-Diffusion, Face-Fusion.
4,443 starsC++GPL-3.0Last push Sep 5, 2026
Rank 4,972: ngxson/wllama
WebAssembly binding for llama.cpp - Enabling on-browser LLM inference
1,341 starsTypeScriptMITLast push Oct 18, 2026
Rank 8,065: hybridgroup/yzma
Go with your own intelligence - Write Go applications that directly integrate llama.cpp for local inference using hardware acceleration on Linux, macOS, Windows, & WebAssembly.
654 starsGoOther licenseLast push Oct 17, 2026