🧠 Large Language Models Fundamentals
🎯 What are Large Language Models?
Large Language Models (LLMs) are deep learning models trained on massive amounts of text data to understand and generate human language. They are based on the Transformer architecture and have billions to trillions of parameters.
Key concept: LLMs learn statistical patterns in language and can perform tasks like text generation, translation, summarization, and question answering.
Deep Explanation
LLMs represent a paradigm shift in AI. Unlike traditional NLP models that are task-specific, LLMs are foundation models that can be adapted to many tasks through prompting and fine-tuning.
Why it matters: LLMs power applications like ChatGPT, Claude, Gemini, and Llama, transforming how we interact with technology.
# Simple LLM interaction (conceptual)
from transformers import AutoModelForCausalLM, AutoTokenizer
# Load a pre-trained LLM
model_name = "gpt2" # Small model for demonstration
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
# Generate text from a prompt
prompt = "The future of artificial intelligence is"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=50)
generated_text = tokenizer.decode(outputs[0])
print(generated_text)
# Output: "The future of artificial intelligence is in the hands of the few..."🤖 Transformer Architecture
🎯 The Transformer Foundation
LLMs are built on the Transformer architecture, introduced in the 2017 paper "Attention Is All You Need". Transformers use self-attention to process sequences in parallel.
Transformer Components
Encoder-Decoder: Original Transformer has both; modern LLMs often use decoder-only. Multi-head attention: Multiple attention heads capture different patterns. Positional encoding: Adds sequence order information. Feed-forward layers: Process attention outputs.
Layer norm: Stabilizes training. Residual connections: Enable deep networks.
# Simplified Transformer block
import torch
import torch.nn as nn
class TransformerBlock(nn.Module):
def __init__(self, d_model, n_heads, d_ff):
super().__init__()
self.attention = nn.MultiheadAttention(d_model, n_heads)
self.ffn = nn.Sequential(
nn.Linear(d_model, d_ff),
nn.ReLU(),
nn.Linear(d_ff, d_model)
)
self.norm1 = nn.LayerNorm(d_model)
self.norm2 = nn.LayerNorm(d_model)
def forward(self, x):
# Self-attention with residual connection
attn_out, _ = self.attention(x, x, x)
x = self.norm1(x + attn_out)
# Feed-forward with residual connection
ffn_out = self.ffn(x)
x = self.norm2(x + ffn_out)
return x
# Decoder-only transformer (GPT-style)
class GPTBlock(nn.Module):
def __init__(self, d_model, n_heads, d_ff):
super().__init__()
self.masked_attention = nn.MultiheadAttention(d_model, n_heads)
self.ffn = nn.Sequential(
nn.Linear(d_model, d_ff),
nn.GELU(),
nn.Linear(d_ff, d_model)
)
self.norm1 = nn.LayerNorm(d_model)
self.norm2 = nn.LayerNorm(d_model)
def forward(self, x, mask):
# Causal (masked) self-attention
attn_out, _ = self.masked_attention(x, x, x, attn_mask=mask)
x = self.norm1(x + attn_out)
ffn_out = self.ffn(x)
x = self.norm2(x + ffn_out)
return x🎯 Attention Mechanisms
🎯 The Core of Transformers
Attention allows the model to focus on relevant parts of the input when generating output. Self-attention considers relationships between all positions in a sequence.
Attention Types
Self-attention: Input attends to itself. Cross-attention: Decoder attends to encoder outputs. Masked attention: Prevents looking ahead in decoding. Multi-head: Multiple attention heads in parallel.
Scaled dot-product: Attention = softmax(QK^T / √d_k) V.
# Attention implementation
import torch
import torch.nn as nn
import torch.nn.functional as F
class ScaledDotProductAttention(nn.Module):
def __init__(self, d_k):
super().__init__()
self.d_k = d_k
def forward(self, Q, K, V, mask=None):
# Scaled dot-product attention
scores = torch.matmul(Q, K.transpose(-2, -1)) / math.sqrt(self.d_k)
if mask is not None:
scores = scores.masked_fill(mask == 0, -1e9)
attention_weights = F.softmax(scores, dim=-1)
output = torch.matmul(attention_weights, V)
return output, attention_weights
class MultiHeadAttention(nn.Module):
def __init__(self, d_model, n_heads):
super().__init__()
self.d_model = d_model
self.n_heads = n_heads
self.d_k = d_model // n_heads
self.W_Q = nn.Linear(d_model, d_model)
self.W_K = nn.Linear(d_model, d_model)
self.W_V = nn.Linear(d_model, d_model)
self.W_O = nn.Linear(d_model, d_model)
self.attention = ScaledDotProductAttention(self.d_k)
def forward(self, Q, K, V, mask=None):
batch_size = Q.size(0)
# Linear projections and reshape
Q = self.W_Q(Q).view(batch_size, -1, self.n_heads, self.d_k).transpose(1, 2)
K = self.W_K(K).view(batch_size, -1, self.n_heads, self.d_k).transpose(1, 2)
V = self.W_V(V).view(batch_size, -1, self.n_heads, self.d_k).transpose(1, 2)
# Apply attention
attn_output, _ = self.attention(Q, K, V, mask)
# Concatenate heads
attn_output = attn_output.transpose(1, 2).contiguous().view(batch_size, -1, self.d_model)
output = self.W_O(attn_output)
return output📚 Pre-training
🎯 Training on Massive Data
Pre-training is the process of training an LLM on a large, diverse corpus of text data (e.g., web pages, books, articles) using self-supervised learning.
Pre-training Objectives
Masked Language Modeling (MLM): Predict masked tokens (BERT). Causal Language Modeling (CLM): Predict next token (GPT). Sequence-to-Sequence: Encoder-decoder pre-training (T5).
Data scale: LLMs are trained on hundreds of billions to trillions of tokens.
# Pre-training objectives (conceptual)
# Masked Language Modeling (MLM)
def mlm_loss(model, input_ids, masked_positions, masked_labels):
# Mask some tokens and predict them
outputs = model(input_ids)
logits = outputs.logits
loss = F.cross_entropy(logits[masked_positions], masked_labels)
return loss
# Causal Language Modeling (CLM)
def clm_loss(model, input_ids, labels):
# Predict next token (shifted right)
outputs = model(input_ids)
logits = outputs.logits
# Shift logits and labels for next-token prediction
shift_logits = logits[..., :-1, :].contiguous()
shift_labels = labels[..., 1:].contiguous()
loss = F.cross_entropy(shift_logits.view(-1, shift_logits.size(-1)), shift_labels.view(-1))
return loss
# Pre-training data pipeline
def preprocess_dataset(documents, tokenizer, max_length=512):
# Tokenize and create training examples
examples = []
for doc in documents:
tokens = tokenizer.encode(doc, max_length=max_length, truncation=True)
examples.append(tokens)
return examples🎯 Fine-tuning
🎯 Adapting to Specific Tasks
Fine-tuning adapts a pre-trained LLM to a specific task using supervised learning on labeled data.
Fine-tuning Methods
Full fine-tuning: Update all parameters. Parameter-Efficient Fine-Tuning (PEFT): Update only a small subset (LoRA, Adapters). Instruction tuning: Fine-tune on instruction-response pairs. RLHF: Reinforcement Learning from Human Feedback.
Transfer learning: Knowledge from pre-training transfers to downstream tasks.
# LoRA (Low-Rank Adaptation) implementation
class LoRA(nn.Module):
def __init__(self, original_layer, rank=8):
super().__init__()
self.original = original_layer
self.rank = rank
self.dim = original_layer.weight.size(0)
# Low-rank matrices
self.A = nn.Parameter(torch.randn(self.dim, rank) * 0.01)
self.B = nn.Parameter(torch.zeros(rank, self.dim))
self.scaling = 1.0 / rank
def forward(self, x):
# Original output + LoRA adaptation
original_out = self.original(x)
lora_out = (x @ self.A.T) @ self.B.T
return original_out + lora_out * self.scaling
# PEFT fine-tuning
def peft_finetune(model, train_data, method='lora', rank=8):
# Apply LoRA to attention layers
for name, module in model.named_modules():
if 'attention' in name.lower():
model._modules[name] = LoRA(module, rank)
# Train only LoRA parameters
trainable_params = [p for p in model.parameters() if p.requires_grad]
optimizer = torch.optim.Adam(trainable_params, lr=1e-4)
# Training loop
for batch in train_data:
loss = compute_loss(model, batch)
loss.backward()
optimizer.step()
optimizer.zero_grad()
return model📊 Scaling Laws
🎯 The Power of Scale
Scaling laws describe how model performance improves with model size, dataset size, and compute.
Scaling Insights
Kaplan scaling laws: Performance scales as a power law with model size, data, and compute. Chinchilla scaling laws: Model size and data should be scaled equally. Emergent abilities: Some abilities only appear at large scales.
Trade-offs: Larger models are more capable but more expensive to train and run.
# Scaling law analysis
def kaplan_scaling(model_size, data_size, compute):
# Approximate performance as function of scale
# Performance ∝ model_size^α * data_size^β * compute^γ
α, β, γ = 0.076, 0.095, 0.090 # Empirical constants
performance = (model_size ** α) * (data_size ** β) * (compute ** γ)
return performance
def chinchilla_optimal(model_size):
# Chinchilla: optimal data size ≈ 20 * model_size
return model_size * 20
def compute_cost(model_size, data_size):
# Compute cost in FLOPs: ≈ 6 * N * D
# N: model parameters, D: data tokens
return 6 * model_size * data_size
# Example: Scaling analysis
model_sizes = [1e9, 10e9, 100e9, 1000e9] # 1B, 10B, 100B, 1T
for size in model_sizes:
optimal_data = chinchilla_optimal(size)
cost = compute_cost(size, optimal_data)
perf = kaplan_scaling(size, optimal_data, cost)
print(f"Model: {size/1e9:.1f}B, Data: {optimal_data/1e9:.1f}B, Cost: {cost/1e18:.2f}E FLOPS")🏛️ Model Architectures
🎯 LLM Architectures
Different LLM architectures include encoder-only, decoder-only, and encoder-decoder models.
Architecture Types
Encoder-only: BERT, RoBERTa (understanding tasks). Decoder-only: GPT, Llama, Claude (generation tasks). Encoder-Decoder: T5, BART (translation, summarization). MoE: Mixture of Experts (sparse computation).
Parameter counts: From 100M to over 1.5T parameters.
# Architecture comparison
class ModelArchitecture:
def __init__(self, name, type, parameters, use_case):
self.name = name
self.type = type
self.parameters = parameters
self.use_case = use_case
architectures = [
ModelArchitecture("BERT", "Encoder-only", "110M-340M", "Classification, QA"),
ModelArchitecture("GPT-3", "Decoder-only", "175B", "Generation, Dialogue"),
ModelArchitecture("T5", "Encoder-Decoder", "11B", "Translation, Summarization"),
ModelArchitecture("LLaMA", "Decoder-only", "7B-70B", "General Purpose"),
ModelArchitecture("Claude", "Decoder-only", "~100B+", "Dialogue, Reasoning"),
ModelArchitecture("GPT-4", "Decoder-only", "~1.5T", "General Purpose")
]
# Mixture of Experts (MoE)
class MoEBlock(nn.Module):
def __init__(self, d_model, d_ff, num_experts=8, num_experts_per_token=2):
super().__init__()
self.experts = nn.ModuleList([
nn.Sequential(
nn.Linear(d_model, d_ff),
nn.GELU(),
nn.Linear(d_ff, d_model)
) for _ in range(num_experts)
])
self.gate = nn.Linear(d_model, num_experts)
self.num_experts_per_token = num_experts_per_token
def forward(self, x):
# Router selects which experts to use per token
gate_logits = self.gate(x)
top_k = torch.topk(gate_logits, self.num_experts_per_token, dim=-1)
weights = F.softmax(top_k.values, dim=-1)
# Compute expert outputs
outputs = torch.zeros_like(x)
for i in range(self.num_experts_per_token):
expert_idx = top_k.indices[..., i]
for e in range(len(self.experts)):
mask = (expert_idx == e)
if mask.any():
expert_out = self.experts[e](x[mask])
outputs[mask] += weights[..., i:i+1][mask] * expert_out
return outputs🔤 Tokenization
🎯 Converting Text to Tokens
Tokenization converts raw text into integer tokens that the model can process. Modern LLMs use subword tokenization for efficiency.
Tokenization Methods
WordPiece: Used by BERT, splits based on subwords. Byte-Pair Encoding (BPE): Used by GPT, merges frequent byte pairs. SentencePiece: Unsupervised tokenizer, no pre-tokenization. Unigram: Probabilistic subword segmentation.
Vocabulary size: Typically 30,000-100,000 tokens.
# BPE tokenization (simplified)
class BPE:
def __init__(self, vocab_size=10000):
self.vocab = {}
self.merges = []
self.vocab_size = vocab_size
def train(self, texts):
# Initialize with character-level tokens
tokens = []
for text in texts:
tokens.append([char for char in text])
# Count pairs and merge
while len(self.vocab) < self.vocab_size:
# Count all adjacent pairs
pair_counts = {}
for token_list in tokens:
for i in range(len(token_list) - 1):
pair = (token_list[i], token_list[i+1])
pair_counts[pair] = pair_counts.get(pair, 0) + 1
if not pair_counts:
break
# Find most frequent pair
best_pair = max(pair_counts, key=pair_counts.get)
# Merge the pair
self.merges.append(best_pair)
self.vocab[self.merge_pair_name(best_pair)] = len(self.vocab)
# Apply merge to all token sequences
tokens = [self.apply_merge(seq, best_pair) for seq in tokens]
def encode(self, text):
# Encode text to tokens
tokens = [char for char in text]
for pair in self.merges:
tokens = self.apply_merge(tokens, pair)
return tokens
def decode(self, tokens):
return ''.join(tokens)
# Example usage
bpe = BPE(vocab_size=100)
bpe.train(["hello world", "hello there", "world peace"])
tokens = bpe.encode("hello")⚡ Inference
🎯 Generating with LLMs
Inference is the process of generating text from an LLM. It involves iterative token generation with various sampling strategies.
Inference Strategies
Greedy decoding: Always choose the most probable token. Beam search: Keep multiple candidate sequences. Sampling: Sample from the probability distribution. Temperature: Controls randomness. Top-k/Top-p: Sampling with constraints.
KV caching: Caching key-value pairs for efficiency.
# Inference strategies
def greedy_decode(model, input_ids, max_length=50):
generated = input_ids
for _ in range(max_length):
outputs = model(generated)
next_logits = outputs.logits[:, -1, :]
next_token = torch.argmax(next_logits, dim=-1)
generated = torch.cat([generated, next_token.unsqueeze(1)], dim=1)
return generated
def beam_search(model, input_ids, beam_size=4, max_length=50):
# Initialize beams
beams = [(input_ids, 0.0)]
for _ in range(max_length):
new_beams = []
for seq, score in beams:
outputs = model(seq)
next_logits = outputs.logits[:, -1, :]
log_probs = F.log_softmax(next_logits, dim=-1)
# Get top-k candidates
top_k = torch.topk(log_probs, beam_size)
for i in range(beam_size):
token = top_k.indices[0, i].unsqueeze(0)
new_seq = torch.cat([seq, token.unsqueeze(1)], dim=1)
new_score = score + top_k.values[0, i].item()
new_beams.append((new_seq, new_score))
# Keep top beams
beams = sorted(new_beams, key=lambda x: x[1], reverse=True)[:beam_size]
return beams[0][0]
def temperature_sampling(model, input_ids, temperature=1.0, max_length=50):
generated = input_ids
for _ in range(max_length):
outputs = model(generated)
logits = outputs.logits[:, -1, :]
# Apply temperature
logits = logits / temperature
probs = F.softmax(logits, dim=-1)
# Sample from distribution
next_token = torch.multinomial(probs, num_samples=1)
generated = torch.cat([generated, next_token], dim=1)
return generated📋 Prompt Engineering
🎯 Crafting Effective Prompts
Prompt engineering is the art of designing inputs that guide LLMs to produce desired outputs. It is the primary way to interact with LLMs.
Prompt Techniques
Zero-shot: No examples, just instructions. Few-shot: Provide examples in the prompt. Chain-of-Thought (CoT): Encourage step-by-step reasoning. Self-consistency: Sample multiple responses and vote. Tree of Thoughts: Explore multiple reasoning paths.
System prompts: Set the model's behavior and role.
# Prompt engineering examples
def zero_shot_prompt(task, input_text):
return f"Task: {task}\nInput: {input_text}\nOutput:"
def few_shot_prompt(task, examples, input_text):
prompt = f"Task: {task}\n\n"
for ex_input, ex_output in examples:
prompt += f"Input: {ex_input}\nOutput: {ex_output}\n\n"
prompt += f"Input: {input_text}\nOutput:"
return prompt
def chain_of_thought_prompt(task, input_text):
return f"Task: {task}\nInput: {input_text}\nLet's think step by step:"
def system_prompt(role, task, input_text):
return f"System: You are {role}.\nUser: {task}\nInput: {input_text}\nAssistant:"
# Example
task = "Classify sentiment as positive or negative"
examples = [
("I love this product!", "positive"),
("This is terrible.", "negative")
]
input_text = "The movie was amazing!"
prompt = few_shot_prompt(task, examples, input_text)
print(prompt)🔄 Alignment
🎯 Making LLMs Safe and Helpful
Alignment ensures that LLMs behave in ways that are helpful, harmless, and honest. It involves training models to follow human preferences.
Alignment Methods
RLHF: Reinforcement Learning from Human Feedback. DPO: Direct Preference Optimization. Constitutional AI: Self-critique and revision. Safety fine-tuning: Training on safety data.
Key challenges: Avoiding harmful outputs, reducing bias, maintaining helpfulness.
# RLHF (conceptual)
class RLHF:
def __init__(self, policy_model, reward_model, ref_model):
self.policy = policy_model
self.reward = reward_model
self.ref = ref_model
self.kl_penalty = 0.1
def generate_response(self, prompt):
return self.policy.generate(prompt)
def compute_reward(self, prompt, response):
# Reward from reward model
r = self.reward(prompt, response)
# KL penalty against reference model
policy_log_probs = self.policy.log_prob(prompt, response)
ref_log_probs = self.ref.log_prob(prompt, response)
kl = policy_log_probs - ref_log_probs
return r - self.kl_penalty * kl
def train_step(self, prompts, responses):
# PPO-style update
rewards = []
for prompt, response in zip(prompts, responses):
rewards.append(self.compute_reward(prompt, response))
# Update policy using rewards
loss = -torch.mean(torch.stack(rewards))
loss.backward()
self.policy.optimizer.step()
# DPO (Direct Preference Optimization)
def dpo_loss(policy, reference, preferred, dispreferred, beta=0.1):
# Preference loss
pref_logits = policy.log_prob(preferred)
dispref_logits = policy.log_prob(dispreferred)
ref_pref_logits = reference.log_prob(preferred)
ref_dispref_logits = reference.log_prob(dispreferred)
diff = (pref_logits - dispref_logits) - (ref_pref_logits - ref_dispref_logits)
loss = -F.logsigmoid(beta * diff)
return loss.mean()📊 Evaluation
🎯 Measuring LLM Performance
LLMs are evaluated on benchmarks that test various capabilities including reasoning, knowledge, and safety.
Evaluation Methods
Perplexity: How well the model predicts text. Benchmarks: MMLU, GSM8K, HumanEval, BIG-bench. Human evaluation: Comparing outputs with human preferences. Adversarial testing: Testing for robustness and safety.
Automatic metrics: BLEU, ROUGE, BERTScore for generation.
# Evaluation metrics
def compute_perplexity(model, text, tokenizer):
# Perplexity = exp(cross_entropy_loss)
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs, labels=inputs.input_ids)
loss = outputs.loss
return torch.exp(loss).item()
def evaluate_on_benchmark(model, benchmark_dataset):
# Run evaluation on a benchmark
results = {}
for task, data in benchmark_dataset.items():
correct = 0
total = len(data)
for example in data:
prediction = model.generate(example['prompt'])
if prediction == example['answer']:
correct += 1
results[task] = correct / total
return results
# BERTScore (semantic similarity)
def bertscore(reference, candidate, model, tokenizer):
# Compute BERTScore for generation evaluation
ref_emb = model.encode(reference)
cand_emb = model.encode(candidate)
# Cosine similarity
similarity = F.cosine_similarity(ref_emb, cand_emb)
return similarity.mean().item()💼 Applications
🎯 Where LLMs Are Used
LLMs power a wide range of applications across business, education, healthcare, and more.
Application Domains
Chatbots: Customer service, virtual assistants. Content creation: Writing, summarization, translation. Education: Tutoring, personalized learning. Healthcare: Clinical documentation, research. Software: Code generation, debugging, documentation.
Examples: ChatGPT, Claude, Gemini, Copilot.
# Application: Customer service chatbot
class CustomerServiceBot:
def __init__(self, llm, system_prompt):
self.llm = llm
self.system_prompt = system_prompt
self.history = []
def respond(self, user_message):
# Add user message to history
self.history.append({"role": "user", "content": user_message})
# Build prompt
messages = [{"role": "system", "content": self.system_prompt}] + self.history
# Generate response
response = self.llm.generate(messages)
# Add response to history
self.history.append({"role": "assistant", "content": response})
return response
# Application: Code generation
def generate_code(llm, task, language="Python"):
prompt = f"Write {language} code for: {task}\n\nCode:"
code = llm.generate(prompt)
return code
# Application: Summarization
def summarize_text(llm, text, max_length=100):
prompt = f"Summarize the following text in {max_length} words:\n\n{text}\n\nSummary:"
summary = llm.generate(prompt)
return summary🚀 Future Directions
🎯 The Future of LLMs
LLM research continues with multimodal models, reasoning improvements, efficiency, and safety.
Emerging Trends
Multimodal: Text + image + audio + video. Reasoning: Improved logical and mathematical reasoning. Efficiency: Smaller, faster models (MoE, distillation). Long context: Processing million-token contexts. Agentic AI: Models that take actions.
Safety: Robust alignment and adversarial robustness.
# Future: Multimodal LLM (conceptual)
class MultimodalLLM:
def __init__(self, text_encoder, vision_encoder, projector, llm):
self.text_encoder = text_encoder
self.vision_encoder = vision_encoder
self.projector = projector
self.llm = llm
def process_image(self, image):
# Encode image and project to text space
image_features = self.vision_encoder(image)
projected_features = self.projector(image_features)
return projected_features
def generate(self, text, image=None):
if image is not None:
image_features = self.process_image(image)
# Combine with text and generate
combined = self.combine_features(text, image_features)
return self.llm.generate(combined)
else:
return self.llm.generate(text)
# Future: Long context (millions of tokens)
def process_long_document(llm, document, query):
# Split document into chunks and process with long context
chunks = split_into_chunks(document)
context = summarize_chunks(llm, chunks)
return llm.generate_with_context(context, query)
# Future: Agentic LLM
class LLMAgent:
def __init__(self, llm, tools):
self.llm = llm
self.tools = tools
self.memory = []
def act(self, task):
# Plan and execute actions
plan = self.llm.plan(task, self.tools)
for action in plan:
result = self.execute_action(action)
self.memory.append(result)
return self.llm.respond(task, self.memory)