InferLLM

Repository facts

Stars

752

Forks

94

Open issues and pull requests

41

Primary language

C++

License

Apache License 2.0 (Apache-2.0)

Owner

MegEngine · Organization

Created

May 4, 2023

Last push

Apr 7, 2024

Topics

• deeplearning
• inference
• llm
• mobile

Rank #7,382 of 9,507 in AI on GitHub, by stars. Data from the GitHub API, fetched Oct 18, 2026; star counts change daily. Stars measure popularity, not quality or safety, and a listing is not an endorsement. How this list is made

Related projects

Projects that share this one’s GitHub topics.

• 6,292

  Rank 6,292: huawei-noah/bolt

  Bolt is a deep learning library with high performance and heterogeneous flexibility.

  955 starsC++MITLast push Apr 11, 2025

• 6,347

  Rank 6,347: uber/neuropod

  A uniform interface to run deep learning models from multiple frameworks

  942 starsC++Apache-2.0Last push Jan 3, 2024

• 5,302

  Rank 5,302: qualcomm/ai-hub-models

  Qualcomm® AI Hub Models is our collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

  1,229 starsPythonBSD-3-ClauseLast push Oct 17, 2026

• 7,497

  Rank 7,497: onepanelio/onepanel

  The open source, end-to-end computer vision platform. Label, build, train, tune, deploy and automate in a unified platform that runs on any cloud and on-premises.

  733 starsGoApache-2.0Last push Feb 25, 2023

• 1,326

  Rank 1,326: cactus-compute/cactus

  Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.

  6,128 starsC++Other licenseLast push Sep 26, 2026

• 4,324

  Rank 4,324: UbiquitousLearning/mllm

  Fast Multimodal LLM on Mobile Devices

  1,628 starsC++MITLast push Sep 30, 2026