native-sparse-attention-pytorch

Repository facts

Stars

816

Forks

51

Open issues and pull requests

12

Primary language

Python

License

MIT License (MIT)

Owner

lucidrains · Personal account

Created

Feb 19, 2025

Last push

Aug 15, 2025

Topics

• artificial-intelligence
• attention
• deep-learning
• sparse-attention

Rank #6,986 of 9,507 in AI on GitHub, by stars. Data from the GitHub API, fetched Oct 18, 2026; star counts change daily. Stars measure popularity, not quality or safety, and a listing is not an endorsement. How this list is made

Related projects

Projects that share this one’s GitHub topics.

• 7,676

  Rank 7,676: svg-project/Sparse-VideoGen

  [ICML2025, NeurIPS2025 Spotlight] Sparse VideoGen 1 & 2: Accelerating Video Diffusion Transformers with Sparse Attention

  710 starsPythonApache-2.0Last push Jul 4, 2026

• 4,536

  Rank 4,536: lucidrains/lambda-networks

  Implementation of LambdaNetworks, a new approach to image recognition that reaches SOTA with less compute

  1,530 starsPythonMITLast push Nov 18, 2020

• 4,804

  Rank 4,804: lucidrains/transfusion-pytorch

  Pytorch implementation of Transfusion, "Predict the Next Token and Diffuse Images with One Multi-Modal Model", from MetaAI

  1,407 starsPythonMITLast push Sep 4, 2026

• 5,360

  Rank 5,360: The-AI-Summer/self-attention-cv

  Implementation of various self-attention mechanisms focused on computer vision. Ongoing repository.

  1,214 starsPythonMITLast push Sep 14, 2021

• 5,476

  Rank 5,476: lucidrains/performer-pytorch

  An implementation of Performer, a linear attention-based transformer, in Pytorch

  1,184 starsPythonMITLast push Feb 2, 2022

• 835

  Rank 835: jadore801120/attention-is-all-you-need-pytorch

  A PyTorch implementation of the Transformer model in "Attention is All You Need".

  9,817 starsPythonMITLast push Apr 16, 2024