[Learn](https://codetrain.ai/learn/) / [How LLMs Actually Work](https://codetrain.ai/learn/llm-foundations/)

# Tokens & embeddings

Turn text into vectors: tokenizers, embeddings, similarity, and position. A class in How LLMs Actually Work. Free to start, no card.

[Start free, no card](https://codetrain.ai/app/) 10 lessons a month on the free tier.

## Curated lessons in this class

### A character tokenizer

python · 5 steps · free

Turn text into integers a model can read, and back again.

1. Play with a tokenizer first
2. Build the vocabulary
3. String -> integer
4. Integer -> string
5. Encode and decode

[Start this lesson](https://codetrain.ai/app/study/?lesson=llm-char-tokenizer)

### Train a mini BPE tokenizer

python · 4 steps · free

Learn a subword vocabulary by merging frequent pairs.

1. Words as lists of symbols
2. Count adjacent pairs
3. Merge the top pair
4. Repeat to grow the vocabulary

[Start this lesson](https://codetrain.ai/app/study/?lesson=llm-bpe)

### Embeddings are a lookup table

python · 3 steps · free

Map token ids to vectors the model can do math on.

1. An embedding table
2. Look up one token
3. Look up a whole sequence

[Start this lesson](https://codetrain.ai/app/study/?lesson=llm-embeddings)

### Cosine similarity

python · 4 steps · free

Measure how close two vectors point, the basis of 'meaning'.

1. The dot product
2. Lengths (norms)
3. Put it together
4. Find the nearest

[Start this lesson](https://codetrain.ai/app/study/?lesson=llm-cosine)

### Positional encoding

python · 4 steps · free

Give an order-blind model a sense of position.

1. Why position needs encoding
2. sin for even, cos for odd
3. The full (seq_len, d) matrix
4. Add it to the embeddings

[Start this lesson](https://codetrain.ai/app/study/?lesson=llm-positional)

## How CodeTrain teaches this

No videos and no multiple-choice quiz at the end. Each lesson is a handful of small steps in a real editor. You type the code yourself, it runs, and the tutor reviews what you actually wrote. It will not hand you the answer, and the step doesn't advance until your code passes. What you get at the end is a record of which steps you wrote unaided, not a completion percentage.

## These curated courses are the starting point, not the product

Everything in this catalog is a **curated** course we wrote by hand. They exist so there is something to start on. What CodeTrain actually does is build a lesson on demand: name **any topic** you want to understand and it writes one for you on the spot, or paste **any public repository** and it teaches you that codebase from the code that is really in it. You are not limited to the list below, and most people stop using it within a session or two.

## The rest of How LLMs Actually Work

[Neural-net foundations](https://codetrain.ai/learn/llm-foundations/llm-foundations-1/)[The Transformer block](https://codetrain.ai/learn/llm-foundations/llm-foundations-3/)[Training, generation & fine-tuning](https://codetrain.ai/learn/llm-foundations/llm-foundations-4/)

[Start this class free](https://codetrain.ai/app/) [Browse every curated course](https://codetrain.ai/learn/)

---

Source: https://codetrain.ai/learn/llm-foundations/llm-foundations-2/

This is the markdown twin of the page above. Every public CodeTrain page serves one on `Accept: text/markdown`. Start at https://codetrain.ai/llms.txt for the full map, or https://codetrain.ai/developers/ for the API.