8. Model Internals¶
The anatomy of a trained model at rest and at run. Children mix static structure (parameters/weights, architecture, layers, neurons, attention heads, vocabulary) with the runtime pipeline (tokenizer → context window → logits → probabilities → sampling). The throughline: text becomes tokens, tokens flow through weights, the model emits logits, sampling turns logits back into a token. Bridges architecture to inference.
Children¶
- parameters / weights
- architecture
- layers
- neurons / units
- activations
- attention heads
- context window
- tokenizer
- vocabulary
- logits
- probabilities
- sampling strategy
Related¶
- Transformer Architecture — the structure these internals instantiate
- Inference — the sampling controls exposed to users
- Math & Data Representation — logits and probability distributions