Rank 5,301: microsoft/MInference
[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to…
1,229 starsPythonMITLast push Oct 10, 2026