halogen-flash-server

Repository facts

Stars

1,018

Forks

75

Open issues and pull requests

22

Primary language

Shell

License

Other license

Owner

peonist-ai · Organization

Created

Aug 26, 2026

Last push

Oct 18, 2026

Topics

• amd
• gfx1151
• gpu-inference
• hip
• inference-engine
• llm
• llm-serving
• local-llm
• long-context
• mixture-of-experts
• openai-api
• qwen
• qwen3
• radeon
• rocm
• ryzen-ai
• speculative-decoding
• strix-halo

Rank #6,051 of 9,507 in AI on GitHub, by stars. Data from the GitHub API, fetched Oct 18, 2026; star counts change daily. Stars measure popularity, not quality or safety, and a listing is not an endorsement. How this list is made

Related projects

Projects that share this one’s GitHub topics.

• 7,888

  Rank 7,888: ARahim3/kaggle-tpu-lab

  Frontier-class open models on a free Kaggle TPU v5e-8: GLM-5.3-Flash 320B MoE (~165 tok/s, our own JAX engine) and Qwen3.8-27B bf16 (~130 tok/s), 262k context, prefix caching. Wor…

  679 starsPythonMITLast push Oct 6, 2026

• 1,372

  Rank 1,372: lemonade-sdk/lemonade

  Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk

  5,927 starsC++Apache-2.0Last push Oct 18, 2026

• 7,564

  Rank 7,564: pegainfer-project/pegainfer

  Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

  724 starsRustApache-2.0Last push Oct 17, 2026

• 2,698

  Rank 2,698: Luce-Org/lucebox

  LLM speculative inference server for heterogeneous hardware & consumer GPUs

  2,928 starsC++Apache-2.0Last push Oct 18, 2026

• 43

  Rank 43: vllm-project/vllm

  A high-throughput and memory-efficient inference and serving engine for LLMs

  94,068 starsPythonApache-2.0Last push Oct 18, 2026

• 6,027

  Rank 6,027: mw00/project-maya

  Project Maya - run GLM-5.3-Flash (321B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-co…

  1,023 starsC++MITLast push Oct 16, 2026