Deepdive-llama3-from-scratch

Repository facts

Stars

629

Forks

52

Open issues and pull requests

0

Primary language

Jupyter Notebook

License

MIT License (MIT)

Owner

therealoliver · Personal account

Created

Feb 19, 2025

Last push

Feb 24, 2025

Topics

• attention
• attention-mechanism
• gpt
• inference
• kv-cache
• language-model
• llama
• llm-configuration
• llms
• mask
• multi-head-attention
• positional-encoding
• residuals
• rms
• rms-norm
• rope
• rotary-position-encoding
• swiglu
• tokenizer
• transformer

Rank #8,265 of 9,507 in AI on GitHub, by stars. Data from the GitHub API, fetched Oct 18, 2026; star counts change daily. Stars measure popularity, not quality or safety, and a listing is not an endorsement. How this list is made

Related projects

Projects that share this one’s GitHub topics.

• 8,585

  Rank 8,585: hirofumi0810/neural_sp

  End-to-end ASR/LM implementation with PyTorch

  595 starsPythonApache-2.0Last push Aug 30, 2021

• 185

  Rank 185: sgl-project/sglang

  SGLang is a high-performance serving framework for large language models and multimodal models.

  37,276 starsPythonApache-2.0Last push Oct 18, 2026

• 551

  Rank 551: BlinkDL/RWKV-LM

  RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combin…

  14,748 starsPythonApache-2.0Last push Sep 28, 2026

• 43

  Rank 43: vllm-project/vllm

  A high-throughput and memory-efficient inference and serving engine for LLMs

  94,068 starsPythonApache-2.0Last push Oct 18, 2026

• 5,672

  Rank 5,672: gordicaleksa/pytorch-original-transformer

  My implementation of the original transformer model (Vaswani et al.). I've additionally included the playground.py file for visualizing otherwise seemingly hard concepts. Currentl…

  1,117 starsJupyter NotebookMITLast push Dec 27, 2020

• 6,178

  Rank 6,178: louisfb01/start-llms

  A complete guide to start and improve your LLM skills in 2026 with little background in the field and stay up-to-date with the latest news and state-of-the-art techniques!

  984 starsMITLast push Jan 23, 2026