Transformers: Understanding the Architecture Behind Modern AI

1. Introduction

Transformers are the heart of modern AI models. AI has seen a lot of breakthrough advancements from ChatGPT to AI, and now. After the development of Transformers, translations and question-answering tasks have seen a breakthrough.

In this blog, I will try to explain Transformers from the basic concepts to how the complete architecture works.

1. Why Transformers?

It's not that Transformers were the first architecture to exist. Various architectures existed before, like RNNs, which take the current input and also the previous input.

For example:

I am a boy

t₁ = I
t₂ = I + am
t₃ = I + am + a
t₄ = I + am + a + boy

But its popularity faded for long sequences as the effect of earlier inputs starts disappearing.

Then the attention mechanism came, which focused not only on previous inputs but also on the relevant inputs to focus on. But their main limitation was that sentences with many words were not handled as effectively.

To overcome the above limitations, Transformers using multi-head attention came into the picture.

1. Transformer: Encoder and Decoder

A Transformer is like a human that first encodes a sentence, understands it, and then decodes it according to the required task.

The Transformer has two major parts:

TRANSFORMER
/ \
/ \
ENCODER DECODER
↓ ↓
Understands Generates
the input output

The original Transformer architecture contains multiple encoder and decoder layers.

The original Transformer uses 6 encoder layers and 6 decoder layers.

1. Tokenization and Embeddings

Before reaching the main architecture, there are some prerequisites.

First, the entire sentence is divided into tokens and then converted into embeddings.

For example:

I am a boy
↓
[I, am, a, boy]
↓
Token IDs
↓
Embedding vectors

Each single token is a vector of a particular dimension, for example (768, 1024) depending on the model.

Think of it like what a particular word means in mathematics in a coordinate system.

So instead of giving the Transformer raw words, we convert every word/token into a numerical vector.

1. Positional Encoding

Since we are working with long sequences, the position of each token is very important.

Which token comes first or second can completely change the meaning of a sentence.

Therefore, positional encodings are used to provide information about the position of each token.

How do they work?

The original Transformer uses sine and cosine functions.

PE(pos, 2i) = sin(pos / 10000^(2i/d_model))

PE(pos, 2i+1) = cos(pos / 10000^(2i/d_model))

For example, let's take:

Sentence = "I love AI"

d_model = 4

For position 0:

PE(0) = [0, 1, 0, 1]

For position 1:

PE(1) ≈ [0.8415, 0.5403, 0.0100, 0.99995]

For position 2:

PE(2) ≈ [0.9093, -0.4161, 0.0200, 0.9998]

Now suppose the embedding of "love" is:

Embedding(love) = [0.4, 0.3, 0.8, 0.2]

Since "love" is at position 1:

Embedding = [0.4000, 0.3000, 0.8000, 0.2000]

PE(1) = [0.8415, 0.5403, 0.0100, 0.99995]

            ↓ ADD

Transformer input = [1.2415, 0.8403, 0.8100, 1.19995]

So positional encoding basically tells the model:

What is the token + Where is the token?

1. Self-Attention

Now comes the most important part: Attention.

Consider the sentence:

I am a boy

Each token is projected into the coordinate system as a vector.

It is broken down into three parts by transformations:

Query (Q)

Key (K)

Value (V)

Think of them like this:

Query

"What am I looking for?"

Key

"See, this is how I look in the coordinate system. Check how much I match with your Query."

Value

"This is what my value in the system contains."

What happens is that the Query vector for each word is multiplied with the Key vectors for each word using a dot product.

The result is divided by √dₖ because for large dimensions the dot product can become large, pushing the probability toward one side.

Then Softmax is applied.

Attention(Q,K,V) = softmax(QKᵀ / √dₖ)V

The Softmax gives the attention weights.

Then these weights are multiplied with the corresponding Value vectors.

This process is repeated multiple times to capture semantic similarity between tokens. This is known as multi-head attention.

1. Numerical Example of Self-Attention

Let's actually calculate a small example.

Consider:

I am a boy

For simplicity, let:

dₖ = 2

Suppose the vectors are:

Token Q K V

I [1,0] [1,1] [1,0]
am [0,1] [1,0] [0,1]
a [1,1] [1,1] [1,1]
boy [1,-1] [0,1] [0,2]

Let's calculate attention for the last token "boy".

Step 1: Calculate Q × Kᵀ

For "boy":

Q(boy) = [1,-1]

Against every Key:

[1,-1] · [1,1] = 0

[1,-1] · [1,0] = 1

[1,-1] · [1,1] = 0

[1,-1] · [0,1] = -1

So:

QKᵀ = [0, 1, 0, -1]

Step 2: Scale by √dₖ

Since:

dₖ = 2
√dₖ = √2 ≈ 1.414

Therefore:

[0, 1, 0, -1] / 1.414

≈ [0, 0.707, 0, -0.707]

Step 3: Apply Softmax

Softmax([0, 0.707, 0, -0.707])

≈ [0.157, 0.256, 0.157, 0.430]

These are the attention weights.

So "boy" pays different amounts of attention to each token.

Step 4: Multiply by V

0.157 × [1,0]
+
0.256 × [0,1]
+
0.157 × [1,1]
+
0.430 × [0,2]

Therefore:

≈ [0.314, 1.017]

So the output representation for "boy" becomes approximately:

[0.314, 1.017]

The important idea is not the particular numbers, but the process:

Q
↓
QKᵀ
↓
Scale by √dₖ
↓
Softmax
↓
Attention weights
↓
Weighted sum of V
↓
Contextual representation

1. Multi-Head Attention

Instead of one attention operation, we run multiple attention heads in parallel.

Each head can learn different types of relationships.

For example, one head may focus on:

subject ↔ verb

while another may focus on:

verb ↔ object

The outputs of all heads are then concatenated and passed through a linear transformation.

Input
│
┌───────────┼───────────┐
↓ ↓ ↓
Head 1 Head 2 Head 3 ... Head h
↓ ↓ ↓
└───────────┼───────────┘
↓
Concatenate
↓
Linear Layer
↓
Output

The equation is:

headᵢ = Attention(QWᵢQ, KWᵢK, VWᵢV)

MultiHead(Q,K,V) =
Concat(head₁, head₂, ..., headₕ)Wᴼ

1. Residual Connection and Layer Normalization

After the attention operation, the output is combined with the original input using a residual connection.

Then layer normalization is applied.

The basic idea is:

Input
↓
Attention
↓
Add original input
↓
Layer Normalization
↓
Output

Residual connections help preserve information and allow gradients to flow through deeper networks.

Layer normalization helps stabilize the network during training.

1. Feed-Forward Neural Network

Further, a Feed-Forward Neural Network (FFN) is applied to each token.

The equation is:

FFN(x) = max(0, xW₁ + b₁)W₂ + b₂

The FFN is applied independently to every token.

Its purpose is to further transform the information received from the self-attention layer and introduce non-linearity.

So a simple way to remember it is:

Attention: Which tokens should interact?

FFN: What transformation should be applied to the resulting representation?

After the FFN, residual connection and layer normalization are applied again.

1. Complete Encoder

The above entire process forms one encoder layer.

Input Embedding
+
Positional Encoding
↓
Multi-Head Self-Attention
↓
Add & Layer Normalization
↓
Feed-Forward Network
↓
Add & Layer Normalization
↓
Encoder Output

The original Transformer has 6 such encoder layers stacked together.

Encoder Layer 1
↓
Encoder Layer 2
↓
Encoder Layer 3
↓
Encoder Layer 4
↓
Encoder Layer 5
↓
Encoder Layer 6
↓
Final Encoder Output

Each layer builds a richer contextual representation.

1. Decoder

After the encoder finishes, the output embedding is passed to the decoder.

After applying positional embeddings, the decoder undergoes masked multi-head attention.

Unlike the encoder, the decoder takes the output token by token at a time.

For example:

At the first step, the decoder predicts the first output token.

Then:

→ I

Then:

I → am

Then:

I am → a

and so on.

This is called autoregressive generation.

1. Why Masked Attention?

The decoder must not see future tokens while generating the current token.

For example, if we are predicting:

I am a boy

while predicting "a", the decoder should not already know "boy".

Therefore, a mask is applied.

I am a boy

I ✓
am ✓ ✓
a ✓ ✓ ✓
boy ✓ ✓ ✓ ✓

This is called causal/masked self-attention.

The target sequence is also right-shifted, meaning the decoder receives previously generated/known tokens to predict the next token.

1. Cross-Attention: How Encoder Output Reaches the Decoder

This is an important part of the Transformer.

The final output of Encoder 6 is used by the cross-attention sublayer of the decoder layers.

In cross-attention:

Query (Q) → comes from decoder

Key (K) → comes from final encoder output

Value (V) → comes from final encoder output

So:

Encoder
↓
Encoder Layer 6
↓
Final Encoder Output
│
┌───────────┼───────────┐
↓ ↓ ↓
Decoder 1 Decoder 2 ... Decoder 6
Cross-Attn Cross-Attn Cross-Attn

The decoder's own output flows from one decoder layer to the next, while the encoder's final representation is available to the cross-attention of every decoder layer.

1. Decoder Layer

A decoder layer therefore contains:

Target Embedding
+
Positional Encoding
↓
Masked Multi-Head Self-Attention
↓
Add & LayerNorm
↓
Multi-Head Cross-Attention
↑
Final Encoder Output
↓
Add & LayerNorm
↓
Feed-Forward Network
↓
Add & LayerNorm
↓
Decoder Output

The original Transformer contains 6 decoder layers.

1. Final Output

After passing through the decoder layers, a linear layer is applied.

Then Softmax converts the final values into probabilities.

Decoder Output
↓
Linear Layer
↓
Softmax
↓
Probability of each token
↓
Most suitable next token

For example:

I am a ______

boy → 0.72
girl → 0.12
student → 0.08
doctor → 0.03
...

The model selects a token according to the generation strategy being used.

1. Complete Transformer Workflow

Here is the complete flow of the original encoder-decoder Transformer:

INPUT SENTENCE
↓
TOKENIZATION
↓
EMBEDDING
↓
POSITIONAL ENCODING
↓
┌──────────────────────┐
│ ENCODER × 6 │
│ │
│ Multi-Head Attention │
│ ↓ │
│ Add & Norm │
│ ↓ │
│ FFN │
│ ↓ │
│ Add & Norm │
└──────────┬───────────┘
↓
FINAL ENCODER OUTPUT
│
│
↓
TARGET → Embedding + Positional Encoding
↓
┌──────────────────────┐
│ DECODER × 6 │
│ │
│ Masked Self-Attention│
│ ↓ │
│ Add & Norm │
│ ↓ │
│ Cross-Attention ←┘
│ ↓
│ Add & Norm
│ ↓
│ FFN
│ ↓
│ Add & Norm
└──────────┬───────────┘
↓
Linear Layer
↓
Softmax
↓
OUTPUT TOKEN
↓
NEXT TOKEN ...

1. Putting the Whole Idea Together

The Transformer can now be understood as a sequence of operations:

Words
↓
Tokens
↓
Embeddings
↓
Positional Information
↓
Self-Attention
↓
Multi-Head Attention
↓
Feed-Forward Network
↓
Repeat through encoder layers
↓
Final Encoder Representation
↓
Decoder
↓
Masked Self-Attention
↓
Cross-Attention with Encoder Output
↓
Feed-Forward Network
↓
Repeat through decoder layers
↓
Linear + Softmax
↓
Output

The most important idea is that attention allows the model to decide which other tokens are relevant when building the representation of a particular token.

1. Applications

Transformers have become extremely important in modern AI.

Some applications include:

Machine translation

Text generation

Question answering

Text summarization

Sentiment analysis

Code generation

Image understanding

Multimodal AI

Models such as BERT, GPT and Vision Transformers are based on the Transformer idea, although their architectures can differ from the original encoder-decoder Transformer.

1. Conclusion

Transformers changed the way we process sequential data by introducing attention as the central mechanism for understanding relationships between tokens.

Instead of processing information strictly one token after another like traditional recurrent architectures, Transformers can use attention to determine which tokens are important to each other.

The major components are:

Tokenization
↓
Embeddings
↓
Positional Encoding
↓
Self-Attention
↓
Multi-Head Attention
↓
Feed-Forward Network
↓
Residual Connections + Layer Normalization
↓
Encoder / Decoder
↓
Cross-Attention
↓
Linear + Softmax
↓
Output

Understanding these components makes it much easier to understand modern architectures such as BERT, GPT and other Transformer-based models.

This is how Transformers work — from converting words into vectors to using attention to understand relationships and finally generating an output.