Learn / How LLMs Actually Work
Tokens & embeddings
Turn text into vectors: tokenizers, embeddings, similarity, and position.
Start free, no card
10 lessons a month on the free tier.
Lessons in this class
A character tokenizer
Turn text into integers a model can read, and back again.
- Play with a tokenizer first
- Build the vocabulary
- String -> integer
- Integer -> string
- Encode and decode
Train a mini BPE tokenizer
Learn a subword vocabulary by merging frequent pairs.
- Words as lists of symbols
- Count adjacent pairs
- Merge the top pair
- Repeat to grow the vocabulary
Embeddings are a lookup table
Map token ids to vectors the model can do math on.
- An embedding table
- Look up one token
- Look up a whole sequence
Cosine similarity
Measure how close two vectors point, the basis of 'meaning'.
- The dot product
- Lengths (norms)
- Put it together
- Find the nearest
Positional encoding
Give an order-blind model a sense of position.
- Why position needs encoding
- sin for even, cos for odd
- The full (seq_len, d) matrix
- Add it to the embeddings
How CodeTrain teaches this
No videos and no multiple-choice quiz at the end. Each lesson is a handful of small steps in a real editor. You type the code yourself, it runs, and the tutor reviews what you actually wrote. It will not hand you the answer, and the step doesn't advance until your code passes. What you get at the end is a record of which steps you wrote unaided, not a completion percentage.