Skip to content

r90941022/LLMs-from-scratch

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

29 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Build a Large Language Model (From Scratch)



In Build a Large Language Model (from Scratch), you'll discover how LLMs work from the inside out. In this book, I'll guide you step by step through creating your own LLM, explaining each stage with clear text, diagrams, and examples.

The method described in this book for training and developing your own small-but-functional model for educational purposes mirrors the approach used in creating large-scale foundational models such as those behind ChatGPT.



Table of Contents

Chapter Main code Code + supplementary
Ch 1: Understanding Large Language Models No code No code
Ch 2: Working with Text Data ch02.ipynb
dataloader.ipynb
./ch02
Ch 3: Understanding Attention Mechanisms ch03.ipynb
multihead-attention.ipynb
./ch03
... ... ...
Appendix A: Introduction to PyTorch code-part1.ipynb
code-part2.ipynb
DDP-script.py
./appendix-A


(A mental model summarizing the contents covered in this book.)

About

Implementing a ChatGPT-like LLM from scratch, step by step

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Jupyter Notebook 93.7%
  • Python 6.3%