Back to Artificial Intelligence & LLMs

LLM Architecture: Transformers & Attention

A deep dive into the components of the Transformer model.

1 week

Topics in this Chapter

1

The Transformer Architecture

Overview of the encoder-decoder stacks that define the model.

2

Self-Attention & Multi-Head Attention

The core mechanism allowing the model to weigh token importance.

3

Positional Encoding

Injecting information about word order into the model.

4

Feed-Forward Networks & Layer Norm

Other essential components of a Transformer block.

GeekDost - Roadmaps & Snippets for Developers