attentions

Repository facts

Stars

548

Forks

73

Open issues and pull requests

2

Primary language

Python

License

MIT License (MIT)

Owner

sooftware · Personal account

Created

Mar 21, 2020

Last push

Mar 4, 2022

Topics

• additive-attention
• attention
• dot-product-attention
• location-aware-attention
• location-sensitive-attension
• multi-head-attention
• pytorch
• relative-multi-head-attention
• relative-positional-encoding

Rank #9,041 of 9,507 in AI on GitHub, by stars. Data from the GitHub API, fetched Oct 18, 2026; star counts change daily. Stars measure popularity, not quality or safety, and a listing is not an endorsement. How this list is made

Related projects

Projects that share this one’s GitHub topics.

• 8,265

  Rank 8,265: therealoliver/Deepdive-llama3-from-scratch

  Achieve the llama3 inference step-by-step, grasp the core concepts, master the process derivation, implement the code.

  629 starsJupyter NotebookMITLast push Feb 24, 2025

• 9,274

  Rank 9,274: anicolson/DeepXi

  Deep Xi: A deep learning approach to a priori SNR estimation implemented in TensorFlow 2/Keras. For speech enhancement and robust ASR.

  525 starsMATLABMPL-2.0Last push Feb 17, 2022

• 544

  Rank 544: kyegomez/OpenMythos

  A theoretical reconstruction of the Claude Mythos architecture, built from first principles using the available research literature.

  14,911 starsPythonMITLast push May 23, 2026

• 659

  Rank 659: xmu-xiaoma666/External-Attention-pytorch

  🍀 Pytorch implementation of various Attention Mechanisms, MLP, Re-parameter, Convolution, which is helpful to further understand papers.⭐⭐⭐

  12,181 starsPythonMITLast push Mar 16, 2026

• 835

  Rank 835: jadore801120/attention-is-all-you-need-pytorch

  A PyTorch implementation of the Transformer model in "Attention is All You Need".

  9,817 starsPythonMITLast push Apr 16, 2024

• 1,719

  Rank 1,719: hyunwoongko/transformer

  Transformer: PyTorch Implementation of "Attention Is All You Need"

  4,672 starsPythonNo license file detectedLast push Jul 15, 2025