attention-transfer

Repository facts

Stars

1,465

Forks

273

Open issues and pull requests

13

Primary language

Jupyter Notebook

License

No license file detected

Owner

szagoruyko · Personal account

Created

Jan 17, 2017

Last push

Jul 11, 2018

Topics

• attention
• deep-learning
• knowledge-distillation
• pytorch

Rank #4,645 of 9,507 in AI on GitHub, by stars. Data from the GitHub API, fetched Oct 18, 2026; star counts change daily. Stars measure popularity, not quality or safety, and a listing is not an endorsement. How this list is made

Related projects

Projects that share this one’s GitHub topics.

• 2,884

  Rank 2,884: gordicaleksa/pytorch-GAT

  My implementation of the original GAT paper (Veličković et al.). I've additionally included the playground.py file for visualizing the Cora dataset, GAT embeddings, an attention m…

  2,688 starsJupyter NotebookMITLast push Nov 17, 2022

• 5,672

  Rank 5,672: gordicaleksa/pytorch-original-transformer

  My implementation of the original transformer model (Vaswani et al.). I've additionally included the playground.py file for visualizing otherwise seemingly hard concepts. Currentl…

  1,117 starsJupyter NotebookMITLast push Dec 27, 2020

• 3,414

  Rank 3,414: alibaba/EasyNLP

  EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit

  2,187 starsPythonApache-2.0Last push Nov 27, 2024

• 4,673

  Rank 4,673: microsoft/NeuronBlocks

  NLP DNN Toolkit - Building Your NLP DNN Models Like Playing Lego

  1,456 starsPythonMITLast push Jul 22, 2023

• 6,540

  Rank 6,540: megvii-research/mdistiller

  The official implementation of [CVPR2022] Decoupled Knowledge Distillation https://arxiv.org/abs/2203.08679 and [ICCV2023] DOT: A Distillation-Oriented Trainer https://openaccess.…

  899 starsPythonNo license file detectedLast push Nov 5, 2023

• 835

  Rank 835: jadore801120/attention-is-all-you-need-pytorch

  A PyTorch implementation of the Transformer model in "Attention is All You Need".

  9,817 starsPythonMITLast push Apr 16, 2024