🧠 Sentiment Analysis Fundamentals

🎯 What is Sentiment Analysis?

Sentiment Analysis (Opinion Mining) is the task of determining the emotional tone behind text — whether it's positive, negative, or neutral. It's one of the most widely used NLP applications, powering product reviews, brand monitoring, customer service, and social media analytics.

Why Sentiment Analysis Matters

Business intelligence: Understand customer opinions at scale. Brand monitoring: Track reputation in real-time. Product feedback: Analyze reviews automatically. Market research: Gauge public opinion. Customer service: Route and prioritize tickets. Foundation: Gateway to advanced NLP.

📌 Opinion Mining📌 Classification📌 NLP📌 Business Value

# Sentiment analysis — the big picture
import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression

# SIMPLE PIPELINE
def sentiment_analysis_pipeline():
    """Complete sentiment analysis workflow."""
    
    # 1. Sample data
    texts = [
        "I love this product! Best purchase ever.",
        "Absolutely terrible. Would not recommend.",
        "It's okay, nothing special.",
        "Amazing quality and fast shipping!",
        "Waste of money. Broke in a week.",
        "Highly recommend to everyone!",
        "Not worth the price at all.",
        "Decent product, does the job.",
    ]
    labels = ['positive', 'negative', 'neutral', 
              'positive', 'negative', 'positive', 
              'negative', 'neutral']
    
    # 2. Vectorize
    vectorizer = TfidfVectorizer(
        ngram_range=(1, 2),
        max_features=1000,
        stop_words='english'
    )
    X = vectorizer.fit_transform(texts)
    
    # 3. Train classifier
    classifier = LogisticRegression(max_iter=1000)
    classifier.fit(X, labels)
    
    # 4. Predict new text
    new_text = ["This product is amazing!"]
    X_new = vectorizer.transform(new_text)
    prediction = classifier.predict(X_new)
    probabilities = classifier.predict_proba(X_new)
    
    return prediction, probabilities

pred, probs = sentiment_analysis_pipeline()
print(f"Prediction: {pred[0]}")
print(f"Probabilities: {probs[0].round(4)}")

# TYPES OF SENTIMENT
print("\n=== Types of Sentiment ===")
print("""
Binary:
  - Positive vs Negative
  - Simple, most common
  - 2 classes

Ternary:
  - Positive, Negative, Neutral
  - More nuanced
  - 3 classes

Fine-grained:
  - Very Negative (-2)
  - Negative (-1)
  - Neutral (0)
  - Positive (+1)
  - Very Positive (+2)
  - 5 or more classes

Continuous:
  - Sentiment score (-1 to +1)
  - Regression task
  - Captures intensity

Multi-dimensional:
  - Emotion (joy, anger, fear, sadness)
  - Intensity (mild, strong)
  - Aspect (what specifically)
""")

# REAL-WORLD DATA
print("\n=== Real-World Applications ===")
print("""
Social Media:
  - Twitter sentiment
  - Facebook comments
  - Reddit discussions
  - Instagram captions

E-commerce:
  - Product reviews
  - Seller ratings
  - Return reasons
  - Q&A analysis

Finance:
  - Stock news sentiment
  - Earnings call analysis
  - Market sentiment
  - Social trading

Healthcare:
  - Patient feedback
  - Drug reviews
  - Mental health monitoring
  - Symptom analysis

Customer Service:
  - Ticket classification
  - Chat analysis
  - Satisfaction surveys
  - Complaint routing

Politics:
  - Voter sentiment
  - Policy reactions
  - Debate analysis
  - Campaign tracking
""")

📊 Levels of Analysis

🎯 Different Granularity

Sentiment analysis operates at multiple levels: document, sentence, aspect, and entity. Each level provides different insights.

Granularity Levels

Document: Whole document sentiment. Sentence: Per-sentence sentiment. Aspect: Sentiment toward specific aspects. Entity: Sentiment toward entities. Fine-grained: Intensity levels. Emotion: Joy, anger, fear, etc.

📄 Document📝 Sentence🎯 Aspect👤 Entity

# Levels of sentiment analysis
import torch
import numpy as np

# 1. DOCUMENT-LEVEL
print("=== Document-Level Sentiment ===")
print("""
Task: Classify entire document (review, article, tweet).

Example:
  "The food was amazing but the service was slow."
  → Overall: Mixed (or negative if service dominates)

Input: Full text
Output: Single sentiment label

Use cases:
  - Product reviews
  - News articles
  - Customer feedback
  - Social media posts

Challenges:
  - Mixed sentiment
  - Long documents
  - Context dependency
  - Topic shifts

Models:
  - Traditional: TF-IDF + SVM/LogReg
  - Deep: LSTM, CNN, BERT
  - Transformer: fine-tuned BERT
""")

# 2. SENTENCE-LEVEL
print("\n=== Sentence-Level Sentiment ===")
print("""
Task: Classify each sentence independently.

Example:
  Sentence 1: "The food was amazing." → Positive
  Sentence 2: "But the service was slow." → Negative
  Sentence 3: "I would still come back." → Positive

Input: Individual sentences
Output: Per-sentence label

Use cases:
  - Review analysis (sentence breakdown)
  - Social media (tweet-level)
  - Dialogue analysis
  - Summarization input

Advantages:
  - Fine-grained
  - Handles mixed sentiment
  - Better for long texts

Challenges:
  - Sentence segmentation
  - Context across sentences
  - Anaphora resolution
""")

# 3. ASPECT-BASED (ABSA)
print("\n=== Aspect-Based Sentiment Analysis ===")
print("""
Task: Sentiment toward specific aspects of an entity.

Example:
  "The food was amazing but the service was slow."
  → Aspect 'food': Positive
  → Aspect 'service': Negative

Output: (aspect, sentiment) pairs

Types:
  - Aspect Term Extraction (ATE)
  - Aspect Category Detection (ACD)
  - Aspect Sentiment Classification (ASC)
  - Aspect-Opinion Pair Extraction (AOPE)

Use cases:
  - Detailed reviews
  - Business intelligence
  - Product improvement
  - Competitive analysis

Models:
  - LSTM + attention
  - BERT-based (ABSA)
  - Span-based models
  - Generative (BART, T5)
""")

# 4. ENTITY-LEVEL
print("\n=== Entity-Level Sentiment ===")
print("""
Task: Sentiment toward specific entities mentioned.

Example:
  "Apple released a new iPhone. Samsung is better."
  → Apple: Neutral
  → Samsung: Positive (comparative)

Output: (entity, sentiment) pairs

Use cases:
  - Brand monitoring
  - Competitive intelligence
  - Named entity sentiment
  - Social listening

Related tasks:
  - NER (Named Entity Recognition)
  - Relation extraction
  - Event extraction
""")

# 5. FINE-GRAINED INTENSITY
print("\n=== Fine-Grained Intensity ===")
print("""
Task: Predict intensity of sentiment.

Scales:
  - 5-point: Very Negative to Very Positive
  - 7-point: More granular
  - Continuous: -1.0 to +1.0

Example:
  "I absolutely love it!" → +2 (very positive)
  "It's okay" → 0 (neutral)
  "Not great" → -1 (mildly negative)
  "Terrible!" → -2 (very negative)

Use cases:
  - Nuanced analysis
  - Ranking systems
  - Trend detection
  - Emotional intensity

Models:
  - Regression (predict score)
  - Ordinal classification
  - Multiclass classification
""")

# 6. EMOTION DETECTION
print("\n=== Emotion Detection ===")
print("""
Task: Identify specific emotions (beyond positive/negative).

Emotions (Ekman's 6):
  - Joy / Happiness
  - Sadness
  - Anger
  - Fear
  - Surprise
  - Disgust

Example:
  "I can't believe they cancelled the show!"
  → Emotion: Anger (not just negative)

Use cases:
  - Mental health monitoring
  - Social media analysis
  - Customer emotion tracking
  - Content moderation

Models:
  - Multi-label classification
  - BERT fine-tuned on emotion datasets
  - GoEmotions dataset (Google)

Datasets:
  - GoEmotions (27 emotions)
  - SemEval-2018 Task 1
  - ISEAR
  - EmoBank
""")

# 7. COMPARISON
print("\n=== Comparison of Levels ===")
print(f"{'Level':<20} {'Granularity':<20} {'Complexity':<15} {'Use Case':<25}")
print("-" * 80)
print(f"{'Document':<20} {'Whole text':<20} {'Low':<15} {'Reviews, news':<25}")
print(f"{'Sentence':<20} {'Per sentence':<20} {'Medium':<15} {'Detailed analysis':<25}")
print(f"{'Aspect':<20} {'Specific features':<20} {'High':<15} {'Product feedback':<25}")
print(f"{'Entity':<20} {'Named entities':<20} {'High':<15} {'Brand monitoring':<25}")
print(f"{'Fine-grained':<20} {'Intensity levels':<20} {'Medium':<15} {'Nuanced sentiment':<25}")
print(f"{'Emotion':<20} {'Specific emotions':<20} {'High':<15} {'Mental health':<25}")

# 8. HIERARCHICAL APPROACH
print("\n=== Hierarchical Approach ===")
print("""
Modern systems combine multiple levels:

Document → Paragraph → Sentence → Aspect → Word

Example pipeline:
  1. Sentence split
  2. Document-level sentiment
  3. Sentence-level sentiment
  4. Aspect extraction
  5. Aspect sentiment
  6. Aggregation

Benefits:
  - Rich understanding
  - Handles complexity
  - Multiple insights
  - Better accuracy

Challenges:
  - More complex
  - More training data
  - Error propagation
  - Slower inference
""")

🧹 Text Preprocessing

🎯 Preparing Text for Analysis

Text preprocessing is crucial for sentiment analysis. Raw text contains noise that hurts model performance. Cleaning it up dramatically improves results.

Preprocessing Steps

Lowercasing: Normalize case. Tokenization: Split into words. Stopwords: Remove common words. Stemming: Reduce to root form. Lemmatization: Dictionary form. Cleaning: Remove noise. Normalization: Standardize text.

🔤 Tokenize📉 Stemming📖 Lemma🧹 Clean

# Text preprocessing for sentiment analysis
import re
import string
import nltk
from nltk.corpus import stopwords
from nltk.stem import PorterStemmer, WordNetLemmatizer
from nltk.tokenize import word_tokenize

# Download NLTK data (one-time)
# nltk.download('punkt')
# nltk.download('stopwords')
# nltk.download('wordnet')

# 1. BASIC CLEANING
def basic_clean(text):
    """Remove noise from text."""
    # Lowercase
    text = text.lower()
    
    # Remove URLs
    text = re.sub(r'http\S+|www\S+', '', text)
    
    # Remove mentions (@user)
    text = re.sub(r'@\w+', '', text)
    
    # Remove hashtags (#tag) — keep the text
    text = re.sub(r'#(\w+)', r'\1', text)
    
    # Remove HTML tags
    text = re.sub(r'<.*?>', '', text)
    
    # Remove punctuation
    text = text.translate(str.maketrans('', '', string.punctuation))
    
    # Remove extra whitespace
    text = ' '.join(text.split())
    
    return text

# 2. TOKENIZATION
def tokenize(text):
    """Split text into tokens."""
    # Simple whitespace
    # tokens = text.split()
    
    # NLTK tokenizer (better)
    tokens = word_tokenize(text)
    
    return tokens

# 3. STOPWORD REMOVAL
def remove_stopwords(tokens):
    """Remove common words."""
    stop_words = set(stopwords.words('english'))
    
    # Keep negation words (important for sentiment!)
    negations = {'not', 'no', 'nor', 'never', 'none', 
                 'nobody', 'nothing', 'neither', 'nowhere',
                 'hardly', 'barely', 'scarcely'}
    stop_words -= negations
    
    return [t for t in tokens if t not in stop_words]

# 4. STEMMING
def stem(tokens):
    """Reduce words to stems."""
    stemmer = PorterStemmer()
    return [stemmer.stem(t) for t in tokens]

# 5. LEMMATIZATION (usually preferred)
def lemmatize(tokens):
    """Reduce words to dictionary form."""
    lemmatizer = WordNetLemmatizer()
    return [lemmatizer.lemmatize(t) for t in tokens]

# 6. HANDLE NEGATIONS (critical for sentiment!)
def handle_negations(tokens):
    """
    Handle negation by prefixing.
    
    Example:
      ['not', 'good'] → ['not_good']
    """
    negations = {'not', 'no', 'never', 'none', 'nothing', 
                 'nobody', 'nowhere', 'neither', 'nor'}
    
    result = []
    i = 0
    while i < len(tokens):
        if tokens[i] in negations and i + 1 < len(tokens):
            # Combine negation with next word
            result.append(f"{tokens[i]}_{tokens[i+1]}")
            i += 2
        else:
            result.append(tokens[i])
            i += 1
    
    return result

# 7. HANDLE EMOJIS
def handle_emojis(text):
    """Convert emojis to text representation."""
    import emoji
    
    # Convert emojis to text: 😊 → :smiling_face:
    text = emoji.demojize(text, delimiters=(" ", " "))
    return text

# 8. HANDLE CONTRACTIONS
def expand_contractions(text):
    """Expand contractions: don't → do not."""
    contractions = {
        "don't": "do not",
        "doesn't": "does not",
        "didn't": "did not",
        "isn't": "is not",
        "aren't": "are not",
        "wasn't": "was not",
        "weren't": "were not",
        "haven't": "have not",
        "hasn't": "has not",
        "hadn't": "had not",
        "won't": "will not",
        "wouldn't": "would not",
        "can't": "cannot",
        "couldn't": "could not",
        "shouldn't": "should not",
        "mightn't": "might not",
        "mustn't": "must not",
        "it's": "it is",
        "i'm": "i am",
        "i've": "i have",
        "i'll": "i will",
        "i'd": "i would",
    }
    
    for contraction, expansion in contractions.items():
        text = text.replace(contraction, expansion)
    
    return text

# 9. COMPLETE PIPELINE
def preprocess_text(text):
    """Complete preprocessing pipeline."""
    # 1. Handle emojis
    text = handle_emojis(text)
    
    # 2. Expand contractions
    text = expand_contractions(text)
    
    # 3. Basic cleaning
    text = basic_clean(text)
    
    # 4. Tokenize
    tokens = tokenize(text)
    
    # 5. Remove stopwords
    tokens = remove_stopwords(tokens)
    
    # 5. Handle negations
    tokens = handle_negations(tokens)
    
    # 6. Lemmatize
    tokens = lemmatize(tokens)
    
    # Return
    return ' '.join(tokens)

# TEST
sample = "I absolutely LOVE this product! 😍 It's amazing. But the shipping was not great... 😞"
print(f"Original: {sample}")
print(f"Processed: {preprocess_text(sample)}")

# PREPROCESSING GUIDELINES
print("\n=== Preprocessing Guidelines ===")
print("""
ESSENTIAL:
  ✓ Lowercasing
  ✓ Remove URLs, mentions
  ✓ Tokenization
  ✓ Handle negations (important!)

RECOMMENDED:
  ✓ Remove stopwords (except negations)
  ✓ Lemmatization (vs stemming)
  ✓ Handle contractions
  ✓ Handle emojis

OPTIONAL:
  ? Remove punctuation (careful!)
  ? Remove numbers
  ? Remove hashtags (keep text)

AVOID:
  ✗ Removing negations (bad for sentiment)
  ✗ Aggressive cleaning (loses info)
  ✗ Removing "not", "very", "really"

KEY INSIGHT:
  Preprocessing is domain-specific!
  Test different approaches.
  What works for one task may not work for another.
""")

# WHY NEGATIONS MATTER
print("\n=== Why Negations Matter ===")
print("""
Example: "This movie was not good"

Without negation handling:
  Tokens: ['movie', 'good']
  Model sees: positive word 'good'
  Prediction: Positive ❌ WRONG!

With negation handling:
  Tokens: ['movie', 'not_good']
  Model sees: negative token 'not_good'
  Prediction: Negative ✅ CORRECT!

Negation words to preserve:
  not, no, never, none, nothing, nobody
  nowhere, neither, nor, hardly, barely
  
Also consider:
  Prefixes: un-, dis-, in-, im-, non-
  Suffixes: -less
  Adverbs: hardly, barely, scarcely
""")

# EMOJI HANDLING
print("\n=== Emoji Handling ===")
print("""
Emojis are crucial for social media sentiment!

Approaches:
  1. Convert to text: 😊 → "smiling_face"
  2. Convert to sentiment: 😊 → +1
  3. Use emoji embeddings
  4. Keep as features

Examples:
  "I love it 😍" → strongly positive
  "Great... 😒" → sarcastic negative
  "Thanks 🙏" → positive
  "Worst day ever 😭" → strongly negative

Common sentiment emojis:
  Positive: 😊 😍 ❤️ 👍 🎉 😂 
  Negative: 😞 😠 😢 👎 💔 😡
  Neutral: 😐 🤔 
""")

📚 Lexicon-Based Methods

🎯 Dictionary-Based Approach

Lexicon-based sentiment analysis uses pre-defined dictionaries of words with sentiment scores. It's simple, interpretable, and requires no training data.

How It Works

1. Lexicon: Dictionary of word → sentiment score. 2. Match: Find sentiment words in text. 3. Aggregate: Sum, average, or weight scores. 4. Handle: Negation, intensity, contrast. 5. Output: Overall sentiment. No training: Works out of the box.

📚 VADER📖 SentiWordNet🎯 AFINN💬 TextBlob

# Lexicon-based sentiment analysis
import re
from collections import defaultdict

# 1. VADER (Valence Aware Dictionary and sEntiment Reasoner)
# Best for social media, handles emojis, slang, intensity
from nltk.sentiment import SentimentIntensityAnalyzer
import nltk
# nltk.download('vader_lexicon')

def analyze_vader(text):
    """VADER sentiment analysis."""
    sia = SentimentIntensityAnalyzer()
    scores = sia.polarity_scores(text)
    
    # scores: {'neg': 0.0, 'neu': 0.5, 'pos': 0.5, 'compound': 0.5}
    # compound: -1 (very negative) to +1 (very positive)
    
    if scores['compound'] >= 0.05:
        label = 'positive'
    elif scores['compound'] <= -0.05:
        label = 'negative'
    else:
        label = 'neutral'
    
    return label, scores

# Test VADER
texts = [
    "I LOVE this product! 😍",
    "Terrible service. Very disappointed.",
    "It's okay, nothing special.",
    "Not bad at all! Actually pretty good.",
    "This is the worst thing ever!!!"
]

for text in texts:
    label, scores = analyze_vader(text)
    print(f"'{text}'")
    print(f"  → {label} (compound: {scores['compound']:.3f})")

# 2. TEXTBLOB
from textblob import TextBlob

def analyze_textblob(text):
    """TextBlob sentiment analysis."""
    blob = TextBlob(text)
    
    # Polarity: -1 to +1
    # Subjectivity: 0 to 1
    polarity = blob.sentiment.polarity
    subjectivity = blob.sentiment.subjectivity
    
    if polarity > 0:
        label = 'positive'
    elif polarity < 0:
        label = 'negative'
    else:
        label = 'neutral'
    
    return label, polarity, subjectivity

# 3. AFINN
# AFINN is a list of words with scores -5 to +5
# Simple to use but no context handling

# 4. SENTIWORDNET
from nltk.corpus import sentiwordnet as swn

def analyze_sentiwordnet(text):
    """SentiWordNet-based analysis."""
    from nltk.tokenize import word_tokenize
    from nltk.tag import pos_tag
    
    tokens = word_tokenize(text.lower())
    tagged = pos_tag(tokens)
    
    pos_score = 0
    neg_score = 0
    
    for word, tag in tagged:
        # Convert POS tag
        if tag.startswith('NN'):
            pos = 'n'
        elif tag.startswith('VB'):
            pos = 'v'
        elif tag.startswith('JJ'):
            pos = 'a'
        elif tag.startswith('RB'):
            pos = 'r'
        else:
            continue
        
        synsets = swn.senti_synsets(word, pos)
        if synsets:
            synset = list(synsets)[0]
            pos_score += synset.pos_score()
            neg_score += synset.neg_score()
    
    if pos_score > neg_score:
        return 'positive'
    elif neg_score > pos_score:
        return 'negative'
    return 'neutral'

# 5. CUSTOM LEXICON
class CustomLexicon:
    """
    Custom lexicon with:
    - Basic sentiment words
    - Negation handling
    - Intensity modifiers
    - Contrast handling
    """
    def __init__(self):
        # Sentiment scores
        self.lexicon = {
            # Positive
            'good': 1, 'great': 2, 'excellent': 3,
            'amazing': 3, 'awesome': 3, 'love': 3,
            'like': 1, 'happy': 2, 'joy': 3,
            'wonderful': 3, 'fantastic': 3, 'best': 3,
            'better': 1, 'positive': 2, 'perfect': 3,
            
            # Negative
            'bad': -1, 'terrible': -3, 'awful': -3,
            'hate': -3, 'dislike': -1, 'sad': -2,
            'worst': -3, 'horrible': -3, 'poor': -2,
            'negative': -2, 'disappointing': -2,
            'broken': -2, 'useless': -3,
            
            # Neutral / Intensifiers
            'very': 1.5, 'extremely': 2, 'really': 1.5,
            'somewhat': 0.5, 'slightly': 0.5,
            'absolutely': 2, 'totally': 1.5,
        }
        
        # Negations
        self.negations = {'not', 'no', 'never', 'none', 
                          'nothing', 'nobody', 'nowhere',
                          'neither', 'nor', 'hardly', 
                          'barely', 'scarcely'}
        
        # Contrast words
        self.contrast = {'but', 'however', 'although', 
                        'though', 'yet', 'nevertheless'}
    
    def analyze(self, text):
        """Analyze sentiment with lexicon."""
        words = text.lower().split()
        
        score = 0
        i = 0
        
        while i < len(words):
            word = words[i]
            
            # Check for negation
            if word in self.negations and i + 1 < len(words):
                next_word = words[i + 1]
                if next_word in self.lexicon:
                    # Flip sentiment
                    score -= self.lexicon[next_word]
                    i += 2
                    continue
            
            # Check for intensifier
            if word in self.lexicon and abs(self.lexicon[word]) > 1:
                if i + 1 < len(words) and words[i + 1] in self.lexicon:
                    # Apply intensity
                    intensity = self.lexicon[word] - 1  # 0.5, 1, 1.5
                    score += self.lexicon[words[i + 1]] * intensity
                    i += 2
                    continue
            
            # Regular sentiment word
            if word in self.lexicon:
                score += self.lexicon[word]
            
            i += 1
        
        if score > 0:
            return 'positive', score
        elif score < 0:
            return 'negative', score
        return 'neutral', score

# Test custom lexicon
lexicon = CustomLexicon()
test_texts = [
    "This is very good!",
    "Not bad at all.",
    "Absolutely terrible.",
    "Really amazing product!",
    "Not very good.",
]

print("\n=== Custom Lexicon ===")
for text in test_texts:
    label, score = lexicon.analyze(text)
    print(f"'{text}' → {label} (score: {score})")

# COMPARISON OF LEXICON METHODS
print("\n=== Lexicon Comparison ===")
print(f"{'Method':<20} {'Training':<15} {'Best For':<30}")
print("-" * 65)
print(f"{'VADER':<20} {'None':<15} {'Social media, slang':<30}")
print(f"{'TextBlob':<20} {'None':<15} {'General text':<30}")
print(f"{'AFINN':<20} {'None':<15} {'Simple sentiment':<30}")
print(f"{'SentiWordNet':<20} {'None':<15} {'Academic, POS-aware':<30}")
print(f"{'Custom':<20} {'None':<15} {'Domain-specific':<30}")

# ADVANTAGES AND LIMITATIONS
print("\n=== Lexicon Methods: Pros and Cons ===")
print("""
ADVANTAGES:
  ✓ No training data needed
  ✓ Fast
  ✓ Interpretable
  ✓ Works for any domain
  ✓ Easy to customize
  ✓ No GPU needed

LIMITATIONS:
  ✗ Cannot learn domain-specific patterns
  ✗ Struggles with sarcasm
  ✗ Context-dependent meanings
  ✗ New slang not covered
  ✗ Word order ignored
  ✗ Negation is hard

WHEN TO USE:
  - Quick prototyping
  - No labeled data
  - Interpretability needed
  - Domain-specific lexicon available
  - Baseline comparison

WHEN TO AVOID:
  - Sarcasm/irony common
  - Domain-specific language
  - High accuracy needed
  - Context matters
""")

🔤 Classical ML (TF-IDF, NB)

🎯 Traditional Machine Learning

Before deep learning, sentiment analysis was dominated by classical ML: TF-IDF features + Naive Bayes, SVM, or Logistic Regression. Still competitive for many tasks.

Classical Pipeline

1. Preprocess: Clean text. 2. Vectorize: TF-IDF or Count. 3. Feature engineering: N-grams, POS. 4. Train: NB, SVM, LogReg. 5. Evaluate: Accuracy, F1. 6. Tune: Grid search.

📊 TF-IDF🎲 Naive Bayes📈 SVM📉 LogReg

# Classical ML for sentiment analysis
import numpy as np
import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer, CountVectorizer
from sklearn.naive_bayes import MultinomialNB, ComplementNB
from sklearn.linear_model import LogisticRegression
from sklearn.svm import LinearSVC
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split, cross_val_score, GridSearchCV
from sklearn.metrics import classification_report, confusion_matrix, accuracy_score

# 1. SAMPLE DATA
texts = [
    "I love this movie, it was fantastic!",
    "Terrible film, waste of time.",
    "Best movie I've seen in years!",
    "Boring and predictable.",
    "Amazing acting and great story.",
    "Awful, don't watch it.",
    "Highly recommend, very entertaining.",
    "Poorly made and disappointing.",
    "Great performances by all actors.",
    "Worst movie ever made.",
    "Not bad, actually enjoyed it.",
    "Really good, loved every minute.",
    "Not worth your time.",
    "One of the best films of the year!",
    "Disappointing, expected more.",
    "Excellent direction and cinematography.",
    "Complete waste of 2 hours.",
    "Beautiful story and great characters.",
    "Terrible acting and bad script.",
    "Masterpiece, absolutely brilliant!",
]
labels = ['pos', 'neg', 'pos', 'neg', 'pos', 'neg', 'pos', 'neg', 
          'pos', 'neg', 'pos', 'pos', 'neg', 'pos', 'neg', 'pos', 
          'neg', 'pos', 'neg', 'pos']

# 2. SPLIT
X_train, X_test, y_train, y_test = train_test_split(
    texts, labels, test_size=0.25, random_state=42, stratify=labels
)

# 3. FEATURE EXTRACTION - TF-IDF
tfidf = TfidfVectorizer(
    ngram_range=(1, 2),      # Unigrams + bigrams
    max_features=5000,       # Top 5000 features
    min_df=1,                # Min document frequency
    max_df=0.9,              # Max document frequency
    sublinear_tf=True,       # Apply sublinear TF scaling
    stop_words='english'     # Remove stopwords
)

X_train_tfidf = tfidf.fit_transform(X_train)
X_test_tfidf = tfidf.transform(X_test)

print(f"TF-IDF features: {X_train_tfidf.shape}")

# 4. TRAIN CLASSIFIERS
classifiers = {
    'Naive Bayes': MultinomialNB(alpha=1.0),
    'Complement NB': ComplementNB(alpha=1.0),
    'Logistic Regression': LogisticRegression(max_iter=1000, C=1.0),
    'Linear SVM': LinearSVC(C=1.0, max_iter=1000),
    'Random Forest': RandomForestClassifier(n_estimators=100, random_state=42)
}

print("\n=== Classifier Comparison ===")
results = {}

for name, clf in classifiers.items():
    # Cross-validation
    cv_scores = cross_val_score(clf, X_train_tfidf, y_train, cv=3)
    
    # Train and test
    clf.fit(X_train_tfidf, y_train)
    y_pred = clf.predict(X_test_tfidf)
    test_acc = accuracy_score(y_test, y_pred)
    
    results[name] = {
        'cv_mean': cv_scores.mean(),
        'cv_std': cv_scores.std(),
        'test_acc': test_acc
    }
    
    print(f"{name}:")
    print(f"  CV: {cv_scores.mean():.4f} (±{cv_scores.std():.4f})")
    print(f"  Test: {test_acc:.4f}")

# 5. FEATURE ENGINEERING
print("\n=== Feature Engineering ===")

# a) N-gram range
vectorizers = {
    'Unigrams': TfidfVectorizer(ngram_range=(1, 1), stop_words='english'),
    'Uni+Bigrams': TfidfVectorizer(ngram_range=(1, 2), stop_words='english'),
    'Uni+Bi+Trigrams': TfidfVectorizer(ngram_range=(1, 3), stop_words='english'),
}

for name, vectorizer in vectorizers.items():
    X_train_v = vectorizer.fit_transform(X_train)
    X_test_v = vectorizer.transform(X_test)
    
    clf = LogisticRegression(max_iter=1000)
    clf.fit(X_train_v, y_train)
    acc = clf.score(X_test_v, y_test)
    
    print(f"{name}: {acc:.4f} ({X_train_v.shape[1]} features)")

# b) Min document frequency
print("\n=== Min DF Impact ===")
for min_df in [1, 2, 3]:
    vectorizer = TfidfVectorizer(min_df=min_df, stop_words='english')
    X_train_v = vectorizer.fit_transform(X_train)
    X_test_v = vectorizer.transform(X_test)
    
    clf = LogisticRegression(max_iter=1000)
    clf.fit(X_train_v, y_train)
    acc = clf.score(X_test_v, y_test)
    
    print(f"min_df={min_df}: {acc:.4f} ({X_train_v.shape[1]} features)")

# c) Max features
print("\n=== Max Features Impact ===")
for max_features in [100, 500, 1000, 5000, None]:
    vectorizer = TfidfVectorizer(
        max_features=max_features, 
        stop_words='english'
    )
    X_train_v = vectorizer.fit_transform(X_train)
    X_test_v = vectorizer.transform(X_test)
    
    clf = LogisticRegression(max_iter=1000)
    clf.fit(X_train_v, y_train)
    acc = clf.score(X_test_v, y_test)
    
    n_features = X_train_v.shape[1] if max_features else 'all'
    print(f"max_features={max_features}: {acc:.4f} ({n_features} features)")

# 6. HYPERPARAMETER TUNING
print("\n=== Hyperparameter Tuning ===")

# Pipeline
from sklearn.pipeline import Pipeline

pipeline = Pipeline([
    ('tfidf', TfidfVectorizer(stop_words='english')),
    ('clf', LogisticRegression(max_iter=1000))
])

# Grid
param_grid = {
    'tfidf__ngram_range': [(1, 1), (1, 2), (1, 3)],
    'tfidf__max_features': [500, 1000, 5000, None],
    'tfidf__min_df': [1, 2, 3],
    'clf__C': [0.1, 1.0, 10.0],
}

# Search
grid_search = GridSearchCV(
    pipeline, param_grid, 
    cv=3, scoring='accuracy', n_jobs=-1
)
grid_search.fit(X_train, y_train)

print(f"Best params: {grid_search.best_params_}")
print(f"Best CV score: {grid_search.best_score_:.4f}")
print(f"Test score: {grid_search.score(X_test, y_test):.4f}")

# 7. CLASSIFICATION REPORT
print("\n=== Classification Report ===")
best_clf = grid_search.best_estimator_
y_pred = best_clf.predict(X_test)

print(classification_report(y_test, y_pred))

# 8. FEATURE IMPORTANCE
print("\n=== Top Features (Logistic Regression) ===")
if hasattr(best_clf.named_steps['clf'], 'coef_'):
    feature_names = best_clf.named_steps['tfidf'].get_feature_names_out()
    coefs = best_clf.named_steps['clf'].coef_[0]
    
    # Top positive features
    top_pos = np.argsort(coefs)[-10:][::-1]
    print("\nTop positive words:")
    for idx in top_pos:
        print(f"  {feature_names[idx]}: {coefs[idx]:.4f}")
    
    # Top negative features
    top_neg = np.argsort(coefs)[:10]
    print("\nTop negative words:")
    for idx in top_neg:
        print(f"  {feature_names[idx]}: {coefs[idx]:.4f}")

# 9. PREDICTION ON NEW TEXT
print("\n=== Predictions on New Text ===")
new_texts = [
    "This movie was absolutely amazing!",
    "Worst film I've ever seen.",
    "It was okay, nothing special.",
    "Not bad, I enjoyed it.",
]

for text in new_texts:
    prediction = best_clf.predict([text])[0]
    print(f"'{text}' → {prediction}")

# 10. ADVANTAGES AND LIMITATIONS
print("\n=== Classical ML: Pros and Cons ===")
print("""
ADVANTAGES:
  ✓ Fast training and inference
  ✓ Works with small datasets
  ✓ Interpretable (feature importance)
  ✓ No GPU needed
  ✓ Good baselines
  ✓ Easy to debug

LIMITATIONS:
  ✗ Cannot capture complex patterns
  ✗ Word order mostly ignored (except n-grams)
  ✗ No semantic understanding
  ✗ Bag-of-words limitation
  ✗ Sarcasm detection fails
  ✗ Context-dependent meaning

WHEN TO USE:
  - Small datasets
  - Fast prototyping
  - Interpretability needed
  - Limited compute
  - Strong baseline

COMPARISON WITH DEEP LEARNING:
  Method          Data Size    Speed    Accuracy
  Classical       Small        Fast     Good
  Deep Learning   Large        Slow     Excellent
  Transformers    Large        Slow     Best
""")

🎯 Word Embeddings

🎯 Dense Vector Representations

Word embeddings map words to dense vectors that capture semantic meaning. They're the foundation of modern NLP, enabling models to understand similarity, analogy, and context.

Embedding Methods

Word2Vec: Skip-gram, CBOW. GloVe: Global vectors. FastText: Subword info. ELMo: Contextual. BERT: Contextual transformer. Usage: Pretrained or trained.

🧠 Word2Vec🌐 GloVe⚡ FastText🤖 BERT

# Word embeddings for sentiment analysis
import numpy as np
import torch
import torch.nn as nn

# 1. WORD2VEC (Pretrained)
from gensim.models import Word2Vec, KeyedVectors

# Load pretrained
try:
    # Google News vectors (300-dim, 3M words)
    w2v_model = KeyedVectors.load_word2vec_format(
        'GoogleNews-vectors-negative300.bin', binary=True
    )
    # Get vector for a word
    vector = w2v_model['good']
    print(f"'good' vector shape: {vector.shape}")
except Exception as e:
    print(f"Pretrained not available: {e}")
    
    # Train your own
    sentences = [
        ['i', 'love', 'this', 'movie'],
        ['this', 'film', 'is', 'terrible'],
        ['great', 'acting', 'and', 'story'],
        ['boring', 'and', 'predictable'],
    ]
    w2v_model = Word2Vec(sentences, vector_size=100, window=5, 
                         min_count=1, workers=4)

# 2. GLOVE (Pretrained)
# Download from: https://nlp.stanford.edu/projects/glove/
try:
    from gensim.scripts.glove2word2vec import glove2word2vec
    from gensim.models import KeyedVectors
    
    # Convert GloVe to Word2Vec format
    glove_input = 'glove.6B.100d.txt'
    word2vec_output = 'glove.6B.100d.word2vec'
    glove2word2vec(glove_input, word2vec_output)
    
    glove_model = KeyedVectors.load_word2vec_format(word2vec_output, binary=False)
    print(f"GloVe loaded")
except Exception as e:
    print(f"GloVe not available: {e}")

# 3. USING EMBEDDINGS FOR SENTIMENT
class SentimentWithEmbeddings(nn.Module):
    """Simple sentiment classifier using word embeddings."""
    def __init__(self, embedding_matrix, hidden_dim=128):
        super().__init__()
        
        vocab_size, embedding_dim = embedding_matrix.shape
        
        # Embedding layer (frozen pretrained)
        self.embedding = nn.Embedding.from_pretrained(
            torch.FloatTensor(embedding_matrix),
            freeze=True  # Don't update embeddings
        )
        
        # Classifier
        self.fc = nn.Sequential(
            nn.Linear(embedding_dim, hidden_dim),
            nn.ReLU(),
            nn.Dropout(0.3),
            nn.Linear(hidden_dim, 2)
        )
    
    def forward(self, x):
        # Average embeddings
        embedded = self.embedding(x)  # (B, seq_len, emb_dim)
        pooled = embedded.mean(dim=1)  # (B, emb_dim)
        return self.fc(pooled)

# 4. BUILD EMBEDDING MATRIX
def build_embedding_matrix(word_to_idx, embedding_model, embedding_dim):
    """Build embedding matrix for vocabulary."""
    vocab_size = len(word_to_idx)
    embedding_matrix = np.zeros((vocab_size, embedding_dim))
    
    for word, idx in word_to_idx.items():
        if word in embedding_model:
            embedding_matrix[idx] = embedding_model[word]
        else:
            # Random for unknown
            embedding_matrix[idx] = np.random.normal(0, 0.1, embedding_dim)
    
    return embedding_matrix

# 5. WORD SIMILARITY
print("\n=== Word Similarity ===")
print("""
Word embeddings capture semantic similarity:

  king - man + woman ≈ queen
  Paris - France + Italy ≈ Rome
  good : bad :: happy : sad

For sentiment:
  - Positive words cluster together
  - Negative words cluster together
  - Similar words have similar vectors
""")

# 6. FASTTEXT (Subword Info)
print("\n=== FastText ===")
print("""
FastText improves on Word2Vec by using subword information:
  - Breaks words into character n-grams
  - Handles rare words better
  - Handles misspellings
  - Good for morphologically rich languages

Example:
  "running" → run, unn, nni, nin, ing, ...
  "runs" → run, uns, ...
  Both share "run" subword

For sentiment:
  - Better on social media (typos, slang)
  - Handles OOV words
  - Good for morphologically rich languages
""")

# 7. CONTEXTUAL EMBEDDINGS
print("\n=== Contextual Embeddings ===")
print("""
Static (Word2Vec, GloVe):
  - One vector per word
  - "bank" always same vector
  - Fast, small
  
Contextual (ELMo, BERT):
  - Different vector per context
  - "bank" in "river bank" vs "money bank"
  - Slower, larger
  - Much better for sentiment

Example:
  "The bank was steep" → river bank
  "I went to the bank" → financial bank
  
Same word, different meaning → different embedding
""")

# 8. USING BERT EMBEDDINGS
from transformers import BertTokenizer, BertModel

def get_bert_embeddings(text):
    """Get BERT contextual embeddings."""
    tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
    model = BertModel.from_pretrained('bert-base-uncased')
    
    # Tokenize
    inputs = tokenizer(text, return_tensors='pt', 
                       padding=True, truncation=True)
    
    # Get embeddings
    with torch.no_grad():
        outputs = model(**inputs)
    
    # Use [CLS] token or mean pooling
    cls_embedding = outputs.last_hidden_state[:, 0, :]
    
    return cls_embedding

# 9. SENTIMENT-SPECIFIC EMBEDDINGS
print("\n=== Sentiment-Specific Embeddings ===")
print("""
Options:
  1. Pretrained general (Word2Vec, GloVe, BERT)
  2. Sentiment-specific (trained on sentiment data)
  3. Fine-tuned embeddings
  4. Task-specific word vectors

Approaches:
  - Fine-tune embeddings on sentiment corpus
  - Train sentiment-specific embeddings
  - Use sentiment lexicon to guide embeddings
  - Multi-task learning with sentiment

Best practices:
  - Start with pretrained general embeddings
  - Fine-tune on sentiment data
  - Or use BERT embeddings (already contextual)
""")

# 10. COMPARISON
print("\n=== Embedding Comparison ===")
print(f"{'Method':<15} {'Dim':<10} {'Context':<15} {'Size':<15} {'Speed':<15}")
print("-" * 70)
print(f"{'Word2Vec':<15} {'100-300':<10} {'Static':<15} {'~1GB':<15} {'Fast':<15}")
print(f"{'GloVe':<15} {'50-300':<10} {'Static':<15} {'~1GB':<15} {'Fast':<15}")
print(f"{'FastText':<15} {'100-300':<10} {'Static':<15} {'~7GB':<15} {'Fast':<15}")
print(f"{'ELMo':<15} {'1024':<10} {'Contextual':<15} {'~350MB':<15} {'Slow':<15}")
print(f"{'BERT':<15} {'768':<10} {'Contextual':<15} {'~440MB':<15} {'Slow':<15}")

# 11. PRACTICAL TIPS
print("\n=== Practical Tips ===")
print("""
1. START WITH PRETRAINED
   - GloVe or Word2Vec for static
   - BERT for contextual
   - Saves training time

2. FINE-TUNE WHEN POSSIBLE
   - Domain-specific data
   - Better results
   - Requires more compute

3. HANDLE OOV WORDS
   - Random vector
   - Or use subword info (FastText)
   - Or character embeddings

4. DIMENSIONALITY
   - 100-300: Standard
   - 768-1024: BERT-style
   - Higher: More capacity, slower

5. FREEZE OR TRAIN
   - Freeze for small data
   - Train for large data
   - Fine-tune for medium

6. COMBINE
   - Static + contextual
   - Multiple embeddings
   - Ensemble

7. EVALUATE
   - Compare on your task
   - Cross-validate
   - Domain-specific metrics
""")

🤖 Deep Learning (LSTM, CNN)

🎯 Neural Networks for Sentiment

Deep learning models like LSTM, CNN, and their combinations dramatically improved sentiment analysis. They learn features automatically and capture complex patterns.

Deep Learning Architectures

CNN: Local n-gram features. LSTM: Sequential, long-range. GRU: Simpler LSTM. BiLSTM: Bidirectional context. Attention: Focus on important parts. BERT: Pretrained transformers.

🧠 CNN🔄 LSTM⚡ GRU🎯 Attention

# Deep learning for sentiment analysis
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.data import Dataset, DataLoader
import numpy as np

# 1. TEXT CNN
class TextCNN(nn.Module):
    """
    CNN for sentiment analysis.
    
    Uses multiple kernel sizes to capture different n-gram features.
    """
    def __init__(self, vocab_size, embed_dim=100, num_classes=2, 
                 num_filters=100, kernel_sizes=[3, 4, 5], dropout=0.5):
        super().__init__()
        
        # Embedding layer
        self.embedding = nn.Embedding(vocab_size, embed_dim, padding_idx=0)
        
        # Convolutional layers with different kernel sizes
        self.convs = nn.ModuleList([
            nn.Conv2d(1, num_filters, (k, embed_dim))
            for k in kernel_sizes
        ])
        
        # Dropout
        self.dropout = nn.Dropout(dropout)
        
        # Classification head
        self.fc = nn.Linear(num_filters * len(kernel_sizes), num_classes)
    
    def forward(self, x):
        # x: (B, seq_len)
        embedded = self.embedding(x)  # (B, seq_len, embed_dim)
        embedded = embedded.unsqueeze(1)  # (B, 1, seq_len, embed_dim)
        
        # Apply convolutions
        conv_outputs = []
        for conv in self.convs:
            conv_out = F.relu(conv(embedded))  # (B, num_filters, seq_len-k+1, 1)
            pooled = F.max_pool2d(conv_out, (conv_out.size(2), 1))  # (B, num_filters, 1, 1)
            conv_outputs.append(pooled.squeeze())
        
        # Concatenate        combined = torch.cat(conv_outputs, dim=1)  # (B, num_filters * num_kernels)
        
        # Classify
        dropped = self.dropout(combined)
        logits = self.fc(dropped)
        
        return logits

# 2. LSTM FOR SENTIMENT
class LSTMClassifier(nn.Module):
    """
    LSTM for sentiment analysis.
    
    Captures sequential dependencies.
    """
    def __init__(self, vocab_size, embed_dim=100, hidden_dim=128,
                 num_layers=2, num_classes=2, dropout=0.5, bidirectional=True):
        super().__init__()
        
        # Embedding
        self.embedding = nn.Embedding(vocab_size, embed_dim, padding_idx=0)
        
        # LSTM
        self.lstm = nn.LSTM(
            embed_dim, hidden_dim,
            num_layers=num_layers,
            bidirectional=bidirectional,
            dropout=dropout if num_layers > 1 else 0,
            batch_first=True
        )
        
        # Classification
        direction = 2 if bidirectional else 1
        self.fc = nn.Sequential(
            nn.Linear(hidden_dim * direction, hidden_dim),
            nn.ReLU(),
            nn.Dropout(dropout),
            nn.Linear(hidden_dim, num_classes)
        )
    
    def forward(self, x):
        # x: (B, seq_len)
        embedded = self.embedding(x)  # (B, seq_len, embed_dim)
        
        # LSTM
        lstm_out, (hidden, cell) = self.lstm(embedded)
        # lstm_out: (B, seq_len, hidden_dim * direction)
        
        # Use last hidden state
        if self.lstm.bidirectional:
            # Concatenate forward and backward final states
            hidden_final = torch.cat([hidden[-2], hidden[-1]], dim=1)
        else:
            hidden_final = hidden[-1]
        
        # Classify
        logits = self.fc(hidden_final)
        
        return logits

# 3. BILSTM WITH ATTENTION
class Attention(nn.Module):
    """Attention mechanism for sentiment classification."""
    def __init__(self, hidden_dim):
        super().__init__()
        self.attention = nn.Linear(hidden_dim, 1)
    
    def forward(self, lstm_output):
        # lstm_output: (B, seq_len, hidden_dim)
        scores = self.attention(lstm_output)  # (B, seq_len, 1)
        weights = F.softmax(scores, dim=1)  # (B, seq_len, 1)
        context = (lstm_output * weights).sum(dim=1)  # (B, hidden_dim)
        return context, weights.squeeze(-1)

class BiLSTMAttention(nn.Module):
    """BiLSTM with attention for sentiment."""
    def __init__(self, vocab_size, embed_dim=100, hidden_dim=128,
                 num_classes=2, dropout=0.5):
        super().__init__()
        
        self.embedding = nn.Embedding(vocab_size, embed_dim, padding_idx=0)
        
        self.lstm = nn.LSTM(
            embed_dim, hidden_dim,
            num_layers=1,
            bidirectional=True,
            batch_first=True
        )
        
        self.attention = Attention(hidden_dim * 2)
        
        self.fc = nn.Sequential(
            nn.Linear(hidden_dim * 2, hidden_dim),
            nn.ReLU(),
            nn.Dropout(dropout),
            nn.Linear(hidden_dim, num_classes)
        )
    
    def forward(self, x):
        embedded = self.embedding(x)
        lstm_out, _ = self.lstm(embedded)
        context, weights = self.attention(lstm_out)
        logits = self.fc(context)
        return logits, weights

# 4. GRU (Simpler alternative)
class GRUClassifier(nn.Module):
    """GRU for sentiment analysis (simpler than LSTM)."""
    def __init__(self, vocab_size, embed_dim=100, hidden_dim=128,
                 num_classes=2, dropout=0.5):
        super().__init__()
        
        self.embedding = nn.Embedding(vocab_size, embed_dim, padding_idx=0)
        
        self.gru = nn.GRU(
            embed_dim, hidden_dim,
            num_layers=2,
            bidirectional=True,
            dropout=dropout,
            batch_first=True
        )
        
        self.fc = nn.Linear(hidden_dim * 2, num_classes)
    
    def forward(self, x):
        embedded = self.embedding(x)
        gru_out, hidden = self.gru(embedded)
        
        # Concatenate final states
        hidden_final = torch.cat([hidden[-2], hidden[-1]], dim=1)
        return self.fc(hidden_final)

# 5. TRAINING
def train_model(model, train_loader, val_loader, epochs=10, 
                lr=1e-3, device='cuda'):
    """Train deep learning model."""
    model = model.to(device)
    optimizer = torch.optim.Adam(model.parameters(), lr=lr)
    criterion = nn.CrossEntropyLoss()
    
    for epoch in range(epochs):
        # Training
        model.train()
        train_loss = 0
        train_correct = 0
        train_total = 0
        
        for batch_x, batch_y in train_loader:
            batch_x, batch_y = batch_x.to(device), batch_y.to(device)
            
            optimizer.zero_grad()
            outputs = model(batch_x)
            
            # Handle models with attention
            if isinstance(outputs, tuple):
                outputs = outputs[0]
            
            loss = criterion(outputs, batch_y)
            loss.backward()
            optimizer.step()
            
            train_loss += loss.item()
            _, predicted = outputs.max(1)
            train_total += batch_y.size(0)
            train_correct += predicted.eq(batch_y).sum().item()
        
        # Validation
        model.eval()
        val_correct = 0
        val_total = 0
        
        with torch.no_grad():
            for batch_x, batch_y in val_loader:
                batch_x, batch_y = batch_x.to(device), batch_y.to(device)
                outputs = model(batch_x)
                
                if isinstance(outputs, tuple):
                    outputs = outputs[0]
                
                _, predicted = outputs.max(1)
                val_total += batch_y.size(0)
                val_correct += predicted.eq(batch_y).sum().item()
        
        train_acc = 100 * train_correct / train_total
        val_acc = 100 * val_correct / val_total
        
        print(f"Epoch {epoch+1}: "
              f"Loss={train_loss/len(train_loader):.4f}, "
              f"Train={train_acc:.2f}%, Val={val_acc:.2f}%")

# 6. ARCHITECTURE COMPARISON
print("=== Deep Learning Architectures ===")
print(f"{'Model':<20} {'Params':<12} {'Speed':<12} {'Best For':<25}")
print("-" * 70)
print(f"{'TextCNN':<20} {'~1M':<12} {'Fast':<12} {'n-gram features':<25}")
print(f"{'LSTM':<20} {'~2M':<12} {'Medium':<12} {'Sequential data':<25}")
print(f"{'BiLSTM':<20} {'~4M':<12} {'Slow':<12} {'Bidirectional context':<25}")
print(f"{'BiLSTM+Attn':<20} {'~4M':<12} {'Slow':<12} {'Focus on key parts':<25}")
print(f"{'GRU':<20} {'~3M':<12} {'Medium':<12} {'Faster LSTM':<25}")
print(f"{'BERT':<20} {'~110M':<12} {'Very Slow':<12} {'SOTA accuracy':<25}")

# 7. ADVANTAGES AND LIMITATIONS
print("\n=== Deep Learning: Pros and Cons ===")
print("""
ADVANTAGES:
  ✓ Learn features automatically
  ✓ Capture complex patterns
  ✓ Handle variable-length input
  ✓ Better accuracy than classical
  ✓ Handle context (LSTM, attention)

LIMITATIONS:
  ✗ Require large datasets
  ✗ Computationally expensive
  ✗ Less interpretable
  ✗ Hyperparameter tuning
  ✗ Need GPUs

WHEN TO USE:
  - Large labeled dataset (> 10K samples)
  - Complex patterns
  - High accuracy needed
  - Compute available

COMPARISON:
  Data Size   Best Approach
  < 1K        Classical ML (TF-IDF + SVM)
  1K-10K      Word embeddings + LSTM
  10K-100K    Fine-tuned BERT
  > 100K      BERT or larger transformers
""")

🚀 Transformers (BERT, RoBERTa)

🎯 State-of-the-Art Sentiment Analysis

Transformers like BERT, RoBERTa, and DeBERTa dominate modern sentiment analysis. Pre-trained on massive text, they capture deep language understanding and achieve SOTA with minimal fine-tuning.

Why Transformers Win

Pretrained: Learn from billions of words. Contextual: Word meaning depends on context. Attention: Long-range dependencies. Transfer: Fine-tune for sentiment. SOTA: Best accuracy. Easy: HuggingFace makes it simple.

🤖 BERT🎯 RoBERTa⚡ DistilBERT📈 DeBERTa

# Transformers for sentiment analysis
import torch
from torch.utils.data import Dataset, DataLoader
from transformers import (
    AutoTokenizer, 
    AutoModelForSequenceClassification,
    TrainingArguments,
    Trainer,
    pipeline
)
from sklearn.metrics import accuracy_score, f1_score
import numpy as np

# 1. QUICK START WITH PIPELINE
def quick_sentiment_analysis():
    """Use pretrained pipeline for sentiment."""
    classifier = pipeline(
        'sentiment-analysis',
        model='distilbert-base-uncased-finetuned-sst-2-english'
    )
    
    texts = [
        "I love this product! Best purchase ever.",
        "Terrible. Complete waste of money.",
        "It's okay, nothing special.",
        "Absolutely amazing experience!",
        "Not bad, actually pretty good."
    ]
    
    results = classifier(texts)
    for text, result in zip(texts, results):
        print(f"'{text}'")
        print(f"  → {result['label']} ({result['score']:.4f})")
    
    return results

# 2. FINE-TUNING BERT
class SentimentDataset(Dataset):
    """Custom dataset for sentiment."""
    def __init__(self, texts, labels, tokenizer, max_length=128):
        self.texts = texts
        self.labels = labels
        self.tokenizer = tokenizer
        self.max_length = max_length
    
    def __len__(self):
        return len(self.texts)
    
    def __getitem__(self, idx):
        text = str(self.texts[idx])
        label = self.labels[idx]
        
        encoding = self.tokenizer(
            text,
            add_special_tokens=True,
            max_length=self.max_length,
            padding='max_length',
            truncation=True,
            return_attention_mask=True,
            return_tensors='pt'
        )
        
        return {
            'input_ids': encoding['input_ids'].flatten(),
            'attention_mask': encoding['attention_mask'].flatten(),
            'labels': torch.tensor(label, dtype=torch.long)
        }

def fine_tune_bert(train_texts, train_labels, val_texts, val_labels):
    """Fine-tune BERT for sentiment analysis."""
    
    # Model and tokenizer
    model_name = 'bert-base-uncased'
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    model = AutoModelForSequenceClassification.from_pretrained(
        model_name, num_labels=2
    )
    
    # Datasets
    train_dataset = SentimentDataset(train_texts, train_labels, tokenizer)
    val_dataset = SentimentDataset(val_texts, val_labels, tokenizer)
    
    # Training arguments
    training_args = TrainingArguments(
        output_dir='./results',
        num_train_epochs=3,
        per_device_train_batch_size=16,
        per_device_eval_batch_size=64,
        warmup_steps=500,
        weight_decay=0.01,
        logging_dir='./logs',
        logging_steps=100,
        evaluation_strategy='epoch',
        save_strategy='epoch',
        load_best_model_at_end=True,
        metric_for_best_model='accuracy',
        learning_rate=2e-5,
    )
    
    # Metrics
    def compute_metrics(eval_pred):
        predictions, labels = eval_pred
        predictions = np.argmax(predictions, axis=1)
        return {
            'accuracy': accuracy_score(labels, predictions),
            'f1': f1_score(labels, predictions, average='weighted')
        }
    
    # Trainer
    trainer = Trainer(
        model=model,
        args=training_args,
        train_dataset=train_dataset,
        eval_dataset=val_dataset,
        compute_metrics=compute_metrics,
    )
    
    # Train
    trainer.train()
    
    return model, tokenizer, trainer

# 3. MODEL COMPARISON
print("\n=== Transformer Models for Sentiment ===")
print(f"{'Model':<25} {'Params':<12} {'Speed':<12} {'Accuracy':<12}")
print("-" * 65)
print(f"{'DistilBERT':<25} {'66M':<12} {'Fast':<12} {'Very Good':<12}")
print(f"{'BERT-base':<25} {'110M':<12} {'Medium':<12} {'Excellent':<12}")
print(f"{'RoBERTa-base':<25} {'125M':<12} {'Medium':<12} {'Better':<12}")
print(f"{'DeBERTa-v3-base':<25} {'184M':<12} {'Medium':<12} {'Best (base)':<12}")
print(f"{'ELECTRA-base':<25} {'110M':<12} {'Medium':<12} {'Excellent':<12}")
print(f"{'ALBERT':<25} {'12M':<12} {'Fast':<12} {'Good':<12}")

# 4. PRETRAINED SENTIMENT MODELS
print("\n=== Pretrained Sentiment Models ===")
print("""
DistilBERT SST-2:
  - Trained on Stanford Sentiment Treebank
  - Binary classification
  - 91% accuracy
  - Fast inference

RoBERTa Twitter:
  - Trained on tweets
  - Handles slang, emojis
  - 3-class (pos/neg/neu)
  - cardiffnlp/twitter-roberta-base-sentiment

FinBERT:
  - Financial sentiment
  - Trained on financial news
  - ProsusAI/finbert

Multilingual BERT:
  - 104 languages
  - For cross-lingual sentiment
  - bert-base-multilingual-cased

XLM-RoBERTa:
  - 100 languages
  - Better than mBERT
  - facebook/xlm-roberta-base
""")

# 5. USING PRETRAINED SENTIMENT MODELS
def use_pretrained_sentiment():
    """Use specific sentiment models."""
    
    # Twitter sentiment
    twitter_classifier = pipeline(
        'sentiment-analysis',
        model='cardiffnlp/twitter-roberta-base-sentiment-latest'
    )
    
    tweets = [
        "Just got my new iPhone! Absolutely loving it! 😍",
        "Ugh, worst customer service ever. #annoyed",
        "The movie was meh. Not great, not terrible.",
    ]
    
    print("\n=== Twitter Sentiment ===")
    for tweet in tweets:
        result = twitter_classifier(tweet)[0]
        print(f"'{tweet}'")
        print(f"  → {result['label']} ({result['score']:.4f})")
    
    # Financial sentiment
    try:
        fin_classifier = pipeline(
            'sentiment-analysis',
            model='ProsusAI/finbert'
        )
        
        financial_texts = [
            "The company reported strong earnings, beating expectations.",
            "Stock prices plummeted amid market turmoil.",
            "Revenue remained flat compared to last quarter.",
        ]
        
        print("\n=== Financial Sentiment ===")
        for text in financial_texts:
            result = fin_classifier(text)[0]
            print(f"'{text}'")
            print(f"  → {result['label']} ({result['score']:.4f})")
    except Exception as e:
        print(f"FinBERT not available: {e}")

# 6. ADVANCED: ASPECT-BASED WITH TRANSFORMERS
class ABSAModel(torch.nn.Module):
    """
    Aspect-Based Sentiment Analysis with BERT.
    
    Input: (text, aspect)
    Output: sentiment toward aspect
    """
    def __init__(self, model_name='bert-base-uncased', num_classes=3):
        super().__init__()
        
        self.bert = AutoModelForSequenceClassification.from_pretrained(
            model_name, num_labels=num_classes
        )
    
    def forward(self, input_ids, attention_mask):
        return self.bert(
            input_ids=input_ids, 
            attention_mask=attention_mask
        ).logits

# Format input as: "[CLS] text [SEP] aspect [SEP]"
def prepare_absa_input(text, aspect, tokenizer):
    """Prepare input for ABSA."""
    encoded = tokenizer(
        text,
        aspect,
        add_special_tokens=True,
        max_length=128,
        padding='max_length',
        truncation=True,
        return_tensors='pt'
    )
    return encoded

# 7. HANDLING LONG TEXTS
print("\n=== Handling Long Texts ===")
print("""
BERT has max length 512 tokens. For longer texts:

1. TRUNCATION
   - Simple: truncate to 512
   - Loses information

2. CHUNKING
   - Split into chunks
   - Classify each
   - Aggregate results

3. HIERARCHICAL
   - Sentence-level BERT
   - Document-level aggregation
   - Best for long documents

4. LONG-FORMER MODELS
   - Longformer (4096 tokens)
   - BigBird (4096 tokens)
   - For very long texts

5. SLIDING WINDOW
   - Overlapping windows
   - Aggregate predictions
   - Better coverage

Recommendations:
  - < 512 tokens: Direct BERT
  - 512-2000: Chunking
  - > 2000: Hierarchical or Longformer
""")

# 8. EFFICIENT INFERENCE
print("\n=== Efficient Inference ===")
print("""
Optimization techniques:

1. DISTILLATION
   - DistilBERT (40% smaller, 60% faster)
   - TinyBERT, MobileBERT
   - 97% of BERT performance

2. QUANTIZATION
   - INT8 (4x smaller)
   - Dynamic quantization
   - Minimal accuracy loss

3. PRUNING
   - Remove attention heads
   - Remove layers
   - Sparse models

4. ONNX
   - Optimize for inference
   - 2-3x faster
   - Cross-platform

5. BATCHING
   - Process multiple at once
   - Better GPU utilization
   - Higher throughput

6. CACHING
   - Cache common queries
   - Redis/Memcached
   - Faster responses

Recommendations:
  - Mobile/edge: DistilBERT + quantization
  - Server: BERT/RoBERTa + ONNX
  - High throughput: Batched inference
""")

# 9. SENTIMENT WITH LLMs
print("\n=== Sentiment with LLMs ===")
print("""
Modern approach: Use LLMs via prompting.

Advantages:
  ✓ No training needed
  ✓ Zero-shot or few-shot
  ✓ Handles complex cases
  ✓ Explainable (can ask why)

Approaches:
  1. Zero-shot: "What is the sentiment?"
  2. Few-shot: Provide examples
  3. Chain-of-thought: "Think step by step"
  4. Structured output: JSON format

Example prompt:
  '''
  Analyze the sentiment of this text.
  Return: positive, negative, or neutral.
  Text: {text}
  Sentiment:
  '''

Models:
  - GPT-4 (best but expensive)
  - Claude (great quality)
  - Llama 3 (open, good)
  - Mistral (efficient)
  - Gemini (multimodal)

When to use:
  - Complex/ambiguous texts
  - Need explanations
  - Multi-lingual
  - No training data
  - Low volume

When to avoid:
  - High volume (expensive)
  - Low latency needed
  - Simple texts
  - Privacy concerns
""")

# 10. BEST PRACTICES
print("\n=== Best Practices ===")
print("""
1. START WITH PRETRAINED
   - distilbert-base-uncased-finetuned-sst-2-english
   - Works great out of the box

2. FINE-TUNE FOR DOMAIN
   - 500-5000 labeled examples
   - 3-5 epochs
   - LR: 2e-5

3. HANDLE CLASS IMBALANCE
   - Weighted loss
   - Focal loss
   - Stratified sampling

4. USE PROPER EVALUATION
   - F1 (not just accuracy)
   - Confusion matrix
   - Per-class metrics

5. OPTIMIZE FOR DEPLOYMENT
   - Distillation
   - Quantization
   - ONNX Runtime

6. MONITOR IN PRODUCTION
   - Track accuracy
   - Detect drift
   - Retrain periodically

7. COMBINE APPROACHES
   - Rule-based for edge cases
   - ML for general
   - LLM for complex
""")

🎯 Aspect-Based Sentiment

🎯 Fine-Grained Opinion Mining

Aspect-Based Sentiment Analysis (ABSA) identifies sentiment toward specific aspects of an entity. Instead of "This restaurant is good," it extracts "Food: positive, Service: negative, Price: neutral."

ABSA Tasks

ATE: Extract aspect terms. ACD: Detect aspect categories. ASC: Classify aspect sentiment. AOPE: Extract aspect-opinion pairs. ACSA: Aspect-category sentiment. ASTE: Aspect-sentiment triple extraction.

🎯 Aspect💭 Opinion📊 Sentiment🎭 Triple

# Aspect-Based Sentiment Analysis
import torch
import torch.nn as nn
from transformers import BertModel, BertTokenizer

# 1. EXAMPLE DATA
print("=== ABSA Example ===")
print("""
Text: "The food was delicious but the service was slow."

Aspect Terms: ['food', 'service']
Aspect Categories: ['food quality', 'service']
Opinions: ['delicious', 'slow']
Sentiments: [positive, negative]

Output (Aspect-Sentiment pairs):
  ('food', 'positive')
  ('service', 'negative')

Output (Triples):
  ('food', 'delicious', 'positive')
  ('service', 'slow', 'negative')
""")

# 2. ASPECT TERM EXTRACTION
class AspectTermExtractor:
    """
    Extract aspect terms from text.
    
    Approach: NER-style sequence labeling.
    Tag each token as B-ASP, I-ASP, or O.
    """
    def __init__(self):
        self.tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
        # In practice: fine-tuned BERT for token classification
        from transformers import AutoModelForTokenClassification
        self.model = AutoModelForTokenClassification.from_pretrained(
            'bert-base-uncased', num_labels=3  # B-ASP, I-ASP, O
        )
    
    def extract(self, text):
        """Extract aspect terms."""
        tokens = self.tokenizer(text, return_tensors='pt')
        
        with torch.no_grad():
            outputs = self.model(**tokens)
            predictions = outputs.logits.argmax(-1)[0]
        
        # Decode predictions
        aspect_terms = []
        current_aspect = []
        
        for token, pred in zip(
            self.tokenizer.convert_ids_to_tokens(tokens['input_ids'][0]),
            predictions
        ):
            if pred.item() == 1:  # B-ASP
                if current_aspect:
                    aspect_terms.append(' '.join(current_aspect))
                current_aspect = [token]
            elif pred.item() == 2:  # I-ASP
                current_aspect.append(token)
            else:  # O
                if current_aspect:
                    aspect_terms.append(' '.join(current_aspect))
                    current_aspect = []
        
        if current_aspect:
            aspect_terms.append(' '.join(current_aspect))
        
        return aspect_terms

# 3. ASPECT SENTIMENT CLASSIFICATION
class ASCModel(nn.Module):
    """
    Aspect Sentiment Classification.
    
    Input: (text, aspect)
    Output: sentiment toward aspect
    """
    def __init__(self, num_classes=3, model_name='bert-base-uncased'):
        super().__init__()
        
        self.bert = BertModel.from_pretrained(model_name)
        self.dropout = nn.Dropout(0.3)
        self.classifier = nn.Linear(self.bert.config.hidden_size, num_classes)
    
    def forward(self, input_ids, attention_mask, token_type_ids=None):
        outputs = self.bert(
            input_ids=input_ids,
            attention_mask=attention_mask,
            token_type_ids=token_type_ids
        )
        
        # Use [CLS] token
        pooled = outputs.pooler_output
        dropped = self.dropout(pooled)
        logits = self.classifier(dropped)
        
        return logits

def prepare_asc_input(text, aspect, tokenizer):
    """Prepare input as: [CLS] text [SEP] aspect [SEP]"""
    encoded = tokenizer(
        text,
        aspect,
        add_special_tokens=True,
        max_length=128,
        padding='max_length',
        truncation=True,
        return_tensors='pt'
    )
    return encoded

# 4. ASPECT-OPINION PAIR EXTRACTION (AOPE)
print("\n=== Aspect-Opinion Pair Extraction ===")
print("""
Task: Extract both aspects and their opinions.

Input: "The food was delicious but the service was slow."
Output: [('food', 'delicious'), ('service', 'slow')]

Approaches:
  1. Pipeline: Extract aspects, then opinions
  2. Joint: Extract both together
  3. Span-based: Extract spans
  4. Sequence labeling: Tag each token

Models:
  - BERT + CRF
  - Span-based BERT
  - Generative (BART, T5)
  - GNN-based (dependency parsing)
""")

# 5. ASPECT-SENTIMENT TRIPLE EXTRACTION (ASTE)
print("\n=== ASTE: Aspect-Sentiment Triplet Extraction ===")
print("""
Task: Extract (aspect, opinion, sentiment) triples.

Input: "The food was delicious but the service was slow."
Output: [
  ('food', 'delicious', 'positive'),
  ('service', 'slow', 'negative')
]

This is the most complete ABSA task.

Approaches:
  1. Pipeline: ATE + AOPE + ASC
  2. End-to-end: Joint extraction
  3. Table filling: Grid-based
  4. Span-based: Span classification
  5. Generative: Seq2seq (BART, T5)

Datasets:
  - SemEval 2014/2015/2016
  - ASTE-Data-V2
  - ASAP

Metrics:
  - Precision, Recall, F1
  - Exact match of triples
""")

# 6. HANDS-ON ABSA WITH TRANSFORMERS
print("\n=== Hands-on ABSA ===")
print("""
Using pyabsa library (popular for ABSA):

  from pyabsa import ATEPCCheckpointManager

  # Aspect Term Extraction + Polarity Classification
  extractor = ATEPCCheckpointManager.get_aspect_extractor(
      checkpoint='english', 
      auto_device=True
  )

  text = "The food was delicious but the service was slow."
  result = extractor.extract_aspect(
      inference_source=[text],
      pred_sentiment=True
  )

  # Output:
  # {
  #   'aspect': ['food', 'service'],
  #   'sentiment': ['positive', 'negative']
  # }

Alternative: BERT-based custom

  from transformers import pipeline

  # For aspect + sentiment in one shot:
  # Format: "text [SEP] aspect"
  absa_classifier = pipeline(
      'text-classification',
      model='yangheng/deberta-v3-base-absa-v1.1'
  )

  text = "The food was delicious but the service was slow."
  aspects = ["food", "service"]

  for aspect in aspects:
      result = absa_classifier(f"{text} [SEP] {aspect}")[0]
      print(f"Aspect: {aspect} → {result['label']}")
""")

# 7. ABSA DATASETS
print("\n=== ABSA Datasets ===")
print("""
SemEval-2014 Task 4:
  - Laptop reviews
  - Restaurant reviews
  - Aspect terms + polarity
  - ~6,000 sentences

SemEval-2015 Task 12:
  - Aspect category + opinion
  - Restaurant + laptop

SemEval-2016 Task 5:
  - Multiple domains
  - Multiple languages

MAMS (Multi-Aspect Multi-Sentiment):
  - 12,000+ sentences
  - Multiple aspects per sentence
  - Harder than SemEval

ASTE-Data-V2:
  - 10,000+ sentences
  - Triplet extraction
  - Standard benchmark

ACOS (Amazon):
  - Amazon reviews
  - Aspect-Category-Opinion-Sentiment
  - Quadruple extraction

Datasets available:
  - HuggingFace datasets
  - GitHub repos (ABSA-PyTorch, pyabsa)
""")

# 8. CHALLENGES
print("\n=== ABSA Challenges ===")
print("""
1. MULTIPLE ASPECTS
   - One sentence, many aspects
   - Each with own sentiment
   - Requires disambiguation

2. IMPLICIT ASPECTS
   - Not explicitly mentioned
   - "The price was high" → 'price' aspect implied
   - Harder to detect

3. ASPECT OVERLAP
   - Aspects share words
   - Complex boundary detection

4. LONG DISTANCE
   - Opinion far from aspect
   - "The food, which came after 30 min, was cold"
   - Hard for attention

5. NEGATION
   - "Not delicious" → negative
   - Different from "not delicious" alone

6. CONTRAST
   - "Good but expensive"
   - Multiple polarities

7. DOMAIN SPECIFICITY
   - Aspect definitions vary
   - Restaurant vs laptop vs hotel
""")

# 9. APPLICATIONS
print("\n=== ABSA Applications ===")
print("""
E-COMMERCE:
  - Product review analysis
  - "Battery good, screen bad"
  - Detailed feedback
  - Product improvement

HOSPITALITY:
  - Restaurant reviews
  - "Food great, service slow"
  - Hotel feedback
  - "Room clean, breakfast bad"

AUTOMOTIVE:
  - Car reviews
  - "Engine powerful, interior cheap"
  - Design feedback

FINANCE:
  - Earnings call analysis
  - "Revenue up, guidance down"
  - Stock sentiment

HEALTHCARE:
  - Patient feedback
  - "Doctor good, wait long"
  - Hospital reviews

TECH:
  - Software reviews
  - "UI nice, performance slow"
  - Feature feedback
""")

# 10. BEST PRACTICES
print("\n=== ABSA Best Practices ===")
print("""
1. START WITH PYTORCH/TRANSFORMERS
   - Many pretrained models
   - Custom fine-tuning
   - Well documented

2. USE PYTORCH-BASED LIBRARIES
   - pyabsa (popular)
   - ABSA-PyTorch (original)
   - fastNLP

3. FINETUNE ON DOMAIN DATA
   - 1000-5000 examples
   - 3-5 epochs
   - LR: 1e-5 to 2e-5

4. HANDLE MULTIPLE ASPECTS
   - Separate classification per aspect
   - Or multi-label

5. USE PROPER EVALUATION
   - Exact match (for extraction)
   - F1 per aspect
   - Triplet F1 (for ASTE)

6. PRETRAINED MODELS TO TRY
   - yangheng/deberta-v3-base-absa-v1.1
   - yangheng/deberta-v3-large-absa-v1.1
   - yangheng/t5-base-absa-v1.1

7. DEPLOY
   - Cache aspect predictions
   - Batch processing
   - Quantize for speed
""")

🌍 Multilingual Sentiment

🎯 Sentiment Across Languages

Multilingual sentiment analysis handles text in multiple languages. Approaches range from translation to training multilingual models.

Multilingual Approaches

Translate then analyze: Simple but lossy. Multilingual models: mBERT, XLM-R. Cross-lingual transfer: Train on English, use on others. Language-specific: Dedicated models. Zero-shot: No target language data. Few-shot: Minimal target data.

🌐 mBERT🔄 XLM-R🌍 100+ langs🚀 Zero-shot

# Multilingual sentiment analysis
import torch
from transformers import pipeline, AutoTokenizer, AutoModelForSequenceClassification

# 1. MULTILINGUAL BERT (mBERT)
def mbert_sentiment():
    """Use mBERT for multilingual sentiment."""
    model_name = 'nlptown/bert-base-multilingual-uncased-sentiment'
    
    classifier = pipeline('sentiment-analysis', model=model_name)
    
    texts = [
        "I love this product!",           # English
        "Je déteste ce produit!",         # French
        "Ich liebe dieses Produkt!",      # German
        "¡Odio este producto!",           # Spanish
        "मुझे यह उत्पाद पसंद है!",        # Hindi
        "我喜欢这个产品！",                # Chinese
        "この製品が大好きです！",          # Japanese
    ]
    
    print("=== Multilingual BERT ===")
    for text in texts:
        result = classifier(text)[0]
        print(f"'{text}'")
        print(f"  → {result['label']} ({result['score']:.4f})")

# 2. XLM-ROBERTA
def xlm_roberta_sentiment():
    """Use XLM-RoBERTa for multilingual sentiment."""
    model_name = 'cardiffnlp/twitter-xlm-roberta-base-sentiment'
    
    classifier = pipeline('sentiment-analysis', model=model_name)
    
    tweets = [
        "Just got my new phone! 📱",       # English
        "Ce film est nul 😞",              # French
        "Das Essen ist fantastisch! 🍕",    # German
        "Este producto es terrible 😡",    # Spanish
        "यह बहुत अच्छा है! 😊",            # Hindi
    ]
    
    print("\n=== XLM-RoBERTa ===")
    for tweet in tweets:
        result = classifier(tweet)[0]
        print(f"'{tweet}'")
        print(f"  → {result['label']} ({result['score']:.4f})")

# 3. TRANSLATE + ANALYZE
def translate_and_analyze(texts, source_lang='auto'):
    """Translate to English, then analyze."""
    from transformers import pipeline
    
    # Translation pipeline
    translator = pipeline(
        'translation',
        model='Helsinki-NLP/opus-mt-mul-en'
    )
    
    # Sentiment pipeline
    sentiment = pipeline(
        'sentiment-analysis',
        model='distilbert-base-uncased-finetuned-sst-2-english'
    )
    
    results = []
    for text in texts:
        # Translate
        translated = translator(text)[0]['translation_text']
        
        # Analyze
        sentiment_result = sentiment(translated)[0]
        
        results.append({
            'original': text,
            'translated': translated,
            'sentiment': sentiment_result['label'],
            'confidence': sentiment_result['score']
        })
    
    return results

# 4. LANGUAGE DETECTION
def detect_and_analyze(text):
    """Detect language first, then analyze."""
    from langdetect import detect, DetectorFactory
    
    # For reproducibility
    DetectorFactory.seed = 0
    
    # Detect language
    lang = detect(text)
    
    # Choose model based on language
    models = {
        'en': 'distilbert-base-uncased-finetuned-sst-2-english',
        'de': 'oliverguhr/german-sentiment-bert',
        'fr': 'tblard/tf-allocine',
        'es': 'finiteautomata/beto-sentiment-analysis',
        'it': 'neuraly/bert-base-italian-cased-sentiment',
        'nl': 'wietsedv/bert-base-dutch-cased-finetuned-sentiment',
        'hi': 'nlptown/bert-base-multilingual-uncased-sentiment',
        'zh': 'uer/roberta-base-finetuned-jd-binary-chinese',
    }
    
    if lang not in models:
        # Fall back to multilingual
        model_name = 'nlptown/bert-base-multilingual-uncased-sentiment'
    else:
        model_name = models[lang]
    
    classifier = pipeline('sentiment-analysis', model=model_name)
    result = classifier(text)[0]
    
    return {
        'language': lang,
        'sentiment': result['label'],
        'confidence': result['score']
    }

# 5. LANGUAGES WITH SENTIMENT MODELS
print("\n=== Available Language Models ===")
print("""
WELL-SUPPORTED LANGUAGES:
  English: distilbert, roberta, bert
  German: oliverguhr/german-sentiment-bert
  French: tblard/tf-allocine
  Spanish: finiteautomata/beto-sentiment-analysis
  Italian: neuraly/bert-base-italian-cased-sentiment
  Dutch: wietsedv/bert-base-dutch-cased-finetuned-sentiment
  Chinese: uer/roberta-base-finetuned-jd-binary-chinese
  Japanese: koheiduck/bert-japanese-finetuned-sentiment
  Arabic: CAMeL-Lab/bert-base-arabic-camelbert-da-sentiment
  Hindi: nlptown/bert-base-multilingual-uncased-sentiment
  Russian: blanchefort/rubert-base-cased-sentiment
  Portuguese: rjacob/bert-base-portuguese-cased-sentiment

MULTILINGUAL MODELS (100+ languages):
  - XLM-RoBERTa (facebook)
  - mBERT (Google)
  - XLM (Facebook)
  - LaBSE (Google)

TRANSFER LEARNING:
  - Train on English
  - Zero-shot to other languages
  - Or few-shot adaptation
""")

# 6. CHALLENGES
print("\n=== Multilingual Challenges ===")
print("""
1. LANGUAGE DIVERSITY
   - 7000+ languages
   - Models cover 100-200
   - Low-resource languages underserved

2. CULTURAL CONTEXT
   - "That's sick!" (English slang for "great")
   - "Terrible" might be sarcasm in another culture
   - Slang varies by region

3. SCRIPT DIFFERENCES
   - Latin, Cyrillic, Arabic, Devanagari, etc.
   - Tokenization challenges
   - Subword models help

4. SENTIMENT EXPRESSION
   - Different ways to express sentiment
   - Japanese indirect, English direct
   - Cultural norms

5. TRANSLATION LOSS
   - Idioms don't translate
   - Wordplay
   - Sarcasm
   - Regional expressions

6. CODE-SWITCHING
   - Mixed languages in one text
   - "Ce film is amazing!"
   - Common in social media

7. DOMAIN ADAPTATION
   - Different domains per language
   - Political vs product reviews
""")

# 7. CROSS-LINGUAL TRANSFER
print("\n=== Cross-Lingual Transfer ===")
print("""
Idea: Train on English, apply to other languages.

Methods:
  1. ZERO-SHOT
     - Train English model
     - Direct application to other languages
     - Works surprisingly well with mBERT
     
  2. TRANSLATE-TRAIN
     - Translate training data
     - Train multilingual model
     - Better accuracy
     
  3. TRANSLATE-TEST
     - Translate test data to English
     - Use English model
     - Simple but slow
     
  4. FEW-SHOT ADAPTATION
     - Few examples in target language
     - Fine-tune multilingual model
     - Best results

Results:
  - Zero-shot: 60-80% of monolingual
  - Few-shot (100 examples): 85-95%
  - Full fine-tune: 95-100%
""")

# 8. DEPLOYMENT
print("\n=== Multilingual Deployment ===")
print("""
Strategy 1: Single multilingual model
  - XLM-RoBERTa
  - Works for all languages
  - Larger model (~1GB)
  - Good for many languages

Strategy 2: Language-specific models
  - Best accuracy per language
  - Multiple models
  - More memory
  - Good for few languages

Strategy 3: Hybrid
  - Multilingual base
  - Language-specific heads
  - Efficient
  - Good balance

Strategy 4: Translate + English model
  - Translate to English
  - Use fast English model
  - Translation overhead
  - Simple infrastructure

Recommendation:
  - 1-3 languages: Language-specific
  - 4-20 languages: Multilingual model
  - 20+ languages: Translate + English
  - Very high accuracy: Custom per language
""")

# 9. BEST PRACTICES
print("\n=== Best Practices ===")
print("""
1. DETECT LANGUAGE FIRST
   - langdetect, fastText
   - Choose appropriate model

2. USE PRETRAINED MULTILINGUAL
   - XLM-RoBERTa for many languages
   - Language-specific if available

3. HANDLE CODE-SWITCHING
   - Common in social media
   - Use multilingual model

4. TRANSLATE WHEN NEEDED
   - For language without model
   - Or when English model is best

5. EVALUATE PER LANGUAGE
   - Accuracy varies by language
   - Report per-language metrics

6. CONSIDER CULTURE
   - Sarcasm varies
   - Politeness varies
   - Emoji usage varies

7. PLAN FOR LOW-RESOURCE
   - Few-shot or zero-shot
   - Or accept lower accuracy

8. MONITOR IN PRODUCTION
   - Track per-language performance
   - Detect drift
   - Retrain with new data
""")

😏 Sarcasm & Irony Detection

🎯 Detecting What's Not Said

Sarcasm detection is one of the hardest problems in sentiment analysis. Sarcastic text expresses the opposite of its literal meaning. "Oh great, another flat tire!" is negative, not positive.

Why Sarcasm is Hard

Opposite meaning: Literal ≠ intended. Context-dependent: Needs world knowledge. Cultural: Varies by culture. Signals: Punctuation, emojis, hashtags. Rare: Few examples to learn from. Impact: Reverses sentiment.

🎭 Sarcasm🔄 Irony😏 Signals🎯 Context

# Sarcasm detection
import re
import torch
import torch.nn as nn
from transformers import AutoTokenizer, AutoModelForSequenceClassification

# 1. SARCASM SIGNALS
print("=== Sarcasm Signals ===")
print("""
LINGUISTIC SIGNALS:
  - Hyperbole: "Best day EVER" (when clearly bad)
  - Punctuation: "Great..." (ellipsis)
  - Exclamation: "Oh, fantastic!" (fake enthusiasm)
  - Quotes: "Oh yeah, 'great'"
  - Contrast: "Wonderful, just wonderful" (repetition)
  - Emojis: 🙄 😏 (rolling eyes, smirk)

LEXICAL CUES:
  - "Yeah, right"
  - "As if"
  - "Big deal"
  - "Oh great"
  - "How wonderful"
  - "Just what I needed"

CONTEXTUAL:
  - Known sarcastic user
  - Contradicts other statements
  - Situation-dependent

HASHTAGS (Twitter):
  - #sarcasm
  - #not
  - #irony
  - #sarcastic
  - #justsaying
""")

# 2. RULE-BASED SARCASM DETECTOR
class RuleBasedSarcasmDetector:
    """Simple rule-based sarcasm detection."""
    def __init__(self):
        # Sarcasm signal patterns
        self.sarcasm_patterns = [
            r'\b(yeah|sure|right)\s*,?\s*(right)?\b',
            r'\boh\s+(great|wonderful|fantastic|joy)\b',
            r'\bhow\s+(nice|wonderful|lovely|original)\b',
            r'\bjust\s+(what|great|lovely|wonderful)\b',
            r'\bbig\s+deal\b',
            r'\bas\s+if\b',
            r'\bnot\b.*\b(again|another)\b',
            r'\bso\s+(glad|happy|excited)\b.*\b(not|never)\b',
        ]
        
        # Emoji signals
        self.sarcasm_emojis = ['🙄', '😏', '😒', '🙃', '😐']
        
        # Punctuation signals
        self.ellipsis_pattern = r'\.{2,}'
        self.excessive_punct = r'[!?]{2,}'
        
        # Positive words (used in negative context = sarcasm)
        self.positive_words = [
            'great', 'wonderful', 'amazing', 'fantastic',
            'brilliant', 'perfect', 'excellent', 'lovely'
        ]
        
        # Negative context indicators
        self.negative_context = [
            'broke', 'late', 'cancel', 'fail', 'wrong',
            'problem', 'issue', 'worse', 'bad', 'terrible'
        ]
    
    def detect(self, text):
        """Detect sarcasm in text."""
        text_lower = text.lower()
        signals = []
        
        # 1. Pattern matching
        for pattern in self.sarcasm_patterns:
            if re.search(pattern, text_lower):
                signals.append(f'pattern: {pattern}')
        
        # 2. Emoji signals
        for emoji in self.sarcasm_emojis:
            if emoji in text:
                signals.append(f'emoji: {emoji}')
        
        # 3. Punctuation
        if re.search(self.ellipsis_pattern, text):
            signals.append('ellipsis')
        if re.search(self.excessive_punct, text):
            signals.append('excessive_punctuation')
        
        # 4. Positive word + negative context
        has_positive = any(w in text_lower for w in self.positive_words)
        has_negative = any(w in text_lower for w in self.negative_context)
        if has_positive and has_negative:
            signals.append('positive_word_negative_context')
        
        # 5. Quotes around positive word
        if re.search(r'["\'].*(great|wonderful|nice|good).*["\']', text_lower):
            signals.append('quoted_positive')
        
        # Determine sarcasm
        is_sarcastic = len(signals) >= 1
        
        return {
            'is_sarcastic': is_sarcastic,
            'signals': signals,
            'confidence': min(len(signals) * 0.3, 1.0)
        }

# Test
detector = RuleBasedSarcasmDetector()

texts = [
    "Oh great, another Monday morning...",
    "I absolutely love this product!",
    "Yeah right, like that's going to work",
    "Wonderful, the car broke down again 🙄",
    "Best day ever! (sarcasm)",
    "So glad I have to work on Saturday!",
    "The food was delicious and service was great.",
    "Oh wonderful, more traffic 🙃",
]

print("\n=== Sarcasm Detection ===")
for text in texts:
    result = detector.detect(text)
    status = "😏 SARCASM" if result['is_sarcastic'] else "😊 SINCERE"
    print(f"'{text}'")
    print(f"  → {status}")
    if result['signals']:
        print(f"     Signals: {', '.join(result['signals'])}")

# 3. ML-BASED SARCASM DETECTION
def ml_sarcasm_detection():
    """ML-based sarcasm detection."""
    from transformers import pipeline
    
    # Pretrained sarcasm model
    # Note: Might need to search HuggingFace for sarcasm models
    classifier = pipeline(
        'text-classification',
        model='helinivan/english-sarcasm-detector'
    )
    
    texts = [
        "Oh great, another meeting that could have been an email",
        "I love waiting in line for hours",
        "This is the best day of my life!",
        "Yeah, because that worked so well last time",
    ]
    
    for text in texts:
        result = classifier(text)[0]
        print(f"'{text}'")
        print(f"  → {result['label']} ({result['score']:.4f})")

# 4. SARCASM DATASETS
print("\n=== Sarcasm Datasets ===")
print("""
TWITTER SARCASM:
  - ~20,000 tweets
  - Hashtag: #sarcasm, #sarcastic
  - Self-labeled
  - Good for training

Reddit Sarcasm (SARC):
  - 1.3M sarcastic comments
  - Self-labeled (r/sarcasm)
  - Large, high-quality
  - Standard benchmark

SemEval-2018 Task 3:
  - Irony detection
  - Tweets
  - Binary + multi-class

iSarcasm:
  - 4,000 tweets
  - Re-annotated (better quality)
  - 5 languages
  - More reliable

Sarcasm Corpus V2:
  - Balanced dataset
  - 5,000+ examples
  - Careful annotation

MUStARD:
  - Multimodal sarcasm
  - Video + audio + text
  - Harder task
""")

# 5. IMPACT ON SENTIMENT
print("\n=== Impact on Sentiment ===")
print("""
Sarcasm reverses sentiment polarity:

Without sarcasm detection:
  "Oh great, another flat tire"
  → positive (great) ❌ WRONG

With sarcasm detection:
  → Sarcasm detected
  → Flip sentiment
  → negative ✅ CORRECT

Common sarcastic patterns:

Literal Positive → Actual Negative:
  - "Great, just great" (bad situation)
  - "Wonderful news" (bad news)
  - "I love it" (hate it)
  - "How nice" (not nice)

Literal Negative → Actual Positive:
  - "Not bad at all" (actually good)
  - "Can't complain" (satisfied)
  - "Terrible, just terrible" (jokingly good)

Impact:
  - Can flip 20-30% of predictions
  - Crucial for social media
  - Often overlooked
""")

# 6. APPROACHES COMPARISON
print("\n=== Approaches Comparison ===")
print(f"{'Approach':<25} {'Accuracy':<15} {'Complexity':<15} {'Notes':<25}")
print("-" * 80)
print(f"{'Rule-based':<25} {'~60%':<15} {'Low':<15} {'Fast, no training':<25}")
print(f"{'Classical ML':<25} {'~70%':<15} {'Medium':<15} {'TF-IDF + features':<25}")
print(f"{'Deep Learning':<25} {'~75%':<15} {'High':<15} {'LSTM/CNN':<25}")
print(f"{'BERT-based':<25} {'~85%':<15} {'High':<15} {'SOTA':<25}")
print(f"{'Multimodal':<25} {'~90%':<15} {'Very High':<15} {'Text + image + context':<25}")

# 7. PRACTICAL TIPS
print("\n=== Practical Tips ===")
print("""
1. DETECT BEFORE SENTIMENT
   - First detect sarcasm
   - Then classify sentiment
   - Flip if sarcastic

2. COMBINE APPROACHES
   - Rule-based for obvious cases
   - ML for complex cases
   - Ensemble for robustness

3. USE CONTEXT
   - User history (if available)
   - Conversation context
   - Domain knowledge

4. HANDLE EMOJIS
   - 🙄 😏 😒 → sarcasm signals
   - 😊 ❤️ → sincere
   - Emojis are strong indicators

5. HASHTAGS HELP
   - #sarcasm, #not, #irony
   - Self-labeled data
   - Useful for training

6. ACCEPT LIMITATIONS
   - Sarcasm detection ~85% max
   - Human performance ~90%
   - Perfect detection is impossible

7. MONITOR IMPACT
   - Track sarcasm rate in your data
   - Measure impact on sentiment
   - Decide if worth effort

8. WHEN TO SKIP
   - Formal text (rarely sarcastic)
   - Product reviews (mostly sincere)
   - News articles
   - Low-sarcasm domains
""")

# 8. HANDLING SARCASM IN PRODUCTION
print("\n=== Handling Sarcasm in Production ===")
print("""
Pipeline:
  1. Detect sarcasm (binary classifier)
  2. If sarcastic: flip sentiment
  3. If not: use sentiment directly
  4. Flag low-confidence for review

Confidence thresholds:
  - High confidence: auto-process
  - Medium: flag for review
  - Low: human review

Metrics to track:
  - Sarcasm rate in input
  - Accuracy on sarcastic samples
  - Overall sentiment accuracy
  - False positive rate

Common pitfalls:
  - Over-detecting sarcasm
  - Missing subtle sarcasm
  - Cultural differences
  - Context ignorance

Best practice:
  - Report sarcasm as a feature
  - Let users see what was detected
  - Allow feedback for learning
  - Retrain with new examples
""")

📈 Evaluation Metrics

🎯 Measuring Performance

Proper evaluation is crucial for sentiment analysis. Accuracy alone is misleading for imbalanced data. Use F1, precision, recall, and confusion matrix.

Key Metrics

Accuracy: Correct / total. Precision: True positives / predicted positives. Recall: True positives / actual positives. F1: Harmonic mean. Confusion matrix: Full breakdown. AUC-ROC: Threshold-independent.

📊 Accuracy🎯 F1📉 Precision📈 Recall

# Evaluation metrics for sentiment analysis
import numpy as np
from sklearn.metrics import (
    accuracy_score, precision_score, recall_score, f1_score,
    classification_report, confusion_matrix, roc_auc_score,
    roc_curve, precision_recall_curve, cohen_kappa_score
)
import matplotlib.pyplot as plt
import seaborn as sns

# 1. SAMPLE PREDICTIONS
y_true = np.array([0, 1, 2, 0, 1, 2, 0, 1, 2, 0, 
                   1, 2, 0, 1, 2, 0, 1, 2, 0, 1])
y_pred = np.array([0, 1, 2, 0, 1, 1, 0, 2, 2, 1, 
                   1, 2, 0, 1, 2, 0, 0, 2, 0, 1])

# Labels: 0=negative, 1=neutral, 2=positive
label_names = ['negative', 'neutral', 'positive']

# 2. BASIC METRICS
print("=== Basic Metrics ===")

# Accuracy
accuracy = accuracy_score(y_true, y_pred)
print(f"Accuracy: {accuracy:.4f}")

# Precision, Recall, F1 (weighted)
precision = precision_score(y_true, y_pred, average='weighted')
recall = recall_score(y_true, y_pred, average='weighted')
f1 = f1_score(y_true, y_pred, average='weighted')

print(f"Precision (weighted): {precision:.4f}")
print(f"Recall (weighted): {recall:.4f}")
print(f"F1 (weighted): {f1:.4f}")

# Macro (unweighted)
print(f"\nMacro Precision: {precision_score(y_true, y_pred, average='macro'):.4f}")
print(f"Macro Recall: {recall_score(y_true, y_pred, average='macro'):.4f}")
print(f"Macro F1: {f1_score(y_true, y_pred, average='macro'):.4f}")

# 3. PER-CLASS METRICS
print("\n=== Classification Report ===")
print(classification_report(y_true, y_pred, target_names=label_names))

# 4. CONFUSION MATRIX
print("\n=== Confusion Matrix ===")
cm = confusion_matrix(y_true, y_pred)
print(cm)

# Visualize
plt.figure(figsize=(8, 6))
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues',
            xticklabels=label_names, yticklabels=label_names)
plt.xlabel('Predicted')
plt.ylabel('Actual')
plt.title('Confusion Matrix')
plt.tight_layout()
plt.show()

# 5. BINARY SENTIMENT (POSITIVE VS NEGATIVE)
print("\n=== Binary Sentiment ===")

# Convert to binary (positive vs not)
y_true_binary = (y_true == 2).astype(int)
y_pred_binary = (y_pred == 2).astype(int)

# Metrics
print(f"Accuracy: {accuracy_score(y_true_binary, y_pred_binary):.4f}")
print(f"Precision: {precision_score(y_true_binary, y_pred_binary):.4f}")
print(f"Recall: {recall_score(y_true_binary, y_pred_binary):.4f}")
print(f"F1: {f1_score(y_true_binary, y_pred_binary):.4f}")

# Confusion matrix
cm_binary = confusion_matrix(y_true_binary, y_pred_binary)
tn, fp, fn, tp = cm_binary.ravel()

print(f"\nTrue Negatives: {tn}")
print(f"False Positives: {fp}")
print(f"False Negatives: {fn}")
print(f"True Positives: {tp}")

# 6. ROC AUC (for binary)
if len(np.unique(y_true_binary)) == 2:
    # Simulate probabilities
    y_proba = np.random.rand(len(y_true_binary))
    
    auc = roc_auc_score(y_true_binary, y_proba)
    print(f"\nAUC-ROC: {auc:.4f}")
    
    fpr, tpr, _ = roc_curve(y_true_binary, y_proba)
    
    plt.figure(figsize=(8, 6))
    plt.plot(fpr, tpr, label=f'ROC (AUC = {auc:.3f})')
    plt.plot([0, 1], [0, 1], 'k--', label='Random')
    plt.xlabel('False Positive Rate')
    plt.ylabel('True Positive Rate')
    plt.title('ROC Curve')
    plt.legend()
    plt.grid(alpha=0.3)
    plt.tight_layout()
    plt.show()

# 7. COHEN'S KAPPA
kappa = cohen_kappa_score(y_true, y_pred)
print(f"Cohen's Kappa: {kappa:.4f}")

# 8. INTERPRETING METRICS
print("\n=== Interpreting Metrics ===")
print("""
ACCURACY:
  - Overall correctness
  - Misleading for imbalanced classes
  - Range: [0, 1]

PRECISION:
  - Of predicted positives, how many are correct?
  - Important when false positives are costly
  - Formula: TP / (TP + FP)

RECALL:
  - Of actual positives, how many did we find?
  - Important when false negatives are costly
  - Formula: TP / (TP + FN)

F1:
  - Balance of precision and recall
  - Good for imbalanced data
  - Formula: 2 * P * R / (P + R)

WEIGHTED vs MACRO vs MICRO:
  - Weighted: weighted by class support
  - Macro: unweighted average
  - Micro: global average

COHEN'S KAPPA:
  - Agreement above chance
  - Range: [-1, 1]
  - > 0.8: Excellent
  - > 0.6: Good

CONFUSION MATRIX:
  - Full breakdown
  - Shows which classes confuse
  - Foundation for all metrics
""")

# 9. HANDLING IMBALANCE
print("\n=== Handling Class Imbalance ===")
print("""
For imbalanced sentiment (e.g., 90% positive):

❌ Accuracy: Misleading (90% by predicting all positive)

✅ Better metrics:
  - F1 (macro or weighted)
  - Balanced accuracy
  - Cohen's Kappa
  - Per-class precision/recall

✅ Techniques:
  - Class weights in loss
  - Oversampling minority
  - Undersampling majority
  - SMOTE for text
  - Stratified splits

Example:
  Positive: 900 samples
  Negative: 100 samples
  Ratio: 9:1
  
  A model predicting all positive:
    Accuracy: 90% (misleading!)
    F1 (negative): 0.0
    F1 (macro): 0.47 (more honest)
""")

# 10. CROSS-VALIDATION
print("\n=== Cross-Validation ===")
print("""
For robust evaluation:

1. K-FOLD (K=5 or 10)
   - Split into K folds
   - Train on K-1, test on 1
   - Repeat K times
   - Average metrics

2. STRATIFIED K-FOLD
   - Preserve class ratios
   - Standard for classification

3. REPEATED K-FOLD
   - Multiple runs
   - Better estimate
   - Report mean ± std

4. LEAVE-ONE-OUT
   - For very small datasets
   - N-fold (N = samples)
   - Expensive

Reporting:
  - Mean ± std
  - Confidence intervals
  - Multiple metrics
  - Per-class breakdown

Code:
from sklearn.model_selection import cross_val_score

scores = cross_val_score(
    model, X, y, 
    cv=StratifiedKFold(n_splits=5),
    scoring='f1_macro'
)
print(f"F1: {scores.mean():.4f} ± {scores.std():.4f}")
""")

# 11. STATISTICAL SIGNIFICANCE
print("\n=== Statistical Significance ===")
print("""
When comparing two models:

1. PAIRED T-TEST
   - Same test set
   - Compare per-sample metrics

2. BOOTSTRAP
   - Resample test set
   - Compute confidence intervals
   - Non-parametric

3. McNEMAR'S TEST
   - For paired binary data
   - Compares two classifiers

4. Effect Size
   - Not just significance
   - Practical significance
   - Cohen's d

Reporting:
  - p-value < 0.05 typically
  - Confidence intervals
  - Effect sizes
  - Multiple comparisons correction
""")

# 12. BEST PRACTICES
print("\n=== Best Practices ===")
print("""
1. USE MULTIPLE METRICS
   - Accuracy + F1 + confusion matrix
   - Per-class breakdown
   - Don't rely on one metric

2. HOLD OUT TEST SET
   - Don't tune on test set
   - Use validation for tuning
   - Test only at the end

3. STRATIFIED SPLITS
   - Preserve class distribution
   - Especially for imbalanced

4. CROSS-VALIDATION
   - For robust estimates
   - Report mean ± std

5. PER-CLASS ANALYSIS
   - Which classes are hard?
   - Confusion patterns
   - Error analysis

6. BASELINE COMPARISON
   - Compare to simple baselines
   - Random, majority class
   - Statistical significance

7. ERROR ANALYSIS
   - Manual inspection
   - Pattern discovery
   - Improvement ideas

8. DOMAIN-SPECIFIC METRICS
   - Business metrics matter
   - Cost of errors
   - User satisfaction
""")

# 13. EXAMPLE EVALUATION PIPELINE
def evaluate_model(model, X_test, y_test, label_names):
    """Complete evaluation pipeline."""
    # Predictions
    y_pred = model.predict(X_test)
    
    # Basic metrics
    metrics = {
        'accuracy': accuracy_score(y_test, y_pred),
        'precision_macro': precision_score(y_test, y_pred, average='macro'),
        'recall_macro': recall_score(y_test, y_pred, average='macro'),
        'f1_macro': f1_score(y_test, y_pred, average='macro'),
        'precision_weighted': precision_score(y_test, y_pred, average='weighted'),
        'recall_weighted': recall_score(y_test, y_pred, average='weighted'),
        'f1_weighted': f1_score(y_test, y_pred, average='weighted'),
        'kappa': cohen_kappa_score(y_test, y_pred),
    }
    
    # Print report
    print("=" * 60)
    print("MODEL EVALUATION")
    print("=" * 60)
    for name, value in metrics.items():
        print(f"{name.replace('_', ' ').title()}: {value:.4f}")
    
    print("\n" + "=" * 60)
    print("CLASSIFICATION REPORT")
    print("=" * 60)
    print(classification_report(y_test, y_pred, target_names=label_names))
    
    print("=" * 60)
    print("CONFUSION MATRIX")
    print("=" * 60)
    cm = confusion_matrix(y_test, y_pred)
    print(cm)
    
    return metrics, cm

💼 Real-World Applications

🎯 Sentiment Analysis in Production

Sentiment analysis powers critical business applications across industries. From brand monitoring to customer service, it provides actionable insights at scale.

Application Domains

Social media: Brand monitoring, trend detection. E-commerce: Reviews, product feedback. Finance: News, earnings calls. Healthcare: Patient feedback. Customer service: Ticket routing. Politics: Voter sentiment.

📱 Social🛒 E-commerce💰 Finance🏥 Healthcare

# Real-world applications of sentiment analysis
import numpy as np
import pandas as pd

# 1. SOCIAL MEDIA MONITORING
print("=== Social Media Monitoring ===")
print("""
Applications:
  - Brand sentiment tracking
  - Campaign effectiveness
  - Crisis detection
  - Competitor analysis
  - Influencer evaluation

Data sources:
  - Twitter/X API
  - Reddit API
  - Facebook Graph API
  - Instagram (limited)
  - YouTube comments

Key metrics:
  - Sentiment distribution over time
  - Sentiment by topic/aspect
  - Sentiment by geography
  - Volume + sentiment correlation
  - Sentiment velocity (change rate)

Challenges:
  - Sarcasm and irony
  - Slang and emojis
  - Multiple languages
  - High volume
  - Real-time requirements
  - Bot detection

Tools:
  - Brandwatch
  - Sprout Social
  - Hootsuite Insights
  - Talkwalker
  - Custom pipelines
""")

# 2. E-COMMERCE PRODUCT REVIEWS
print("\n=== E-commerce Product Reviews ===")
print("""
Applications:
  - Product rating prediction
  - Review summarization
  - Feature extraction
  - Fake review detection
  - Seller feedback analysis
  - Recommendation improvement

Tasks:
  1. Overall sentiment classification
  2. Aspect-based analysis
     - Quality, price, shipping, service
  3. Review summarization
     - "Customers love X but complain about Y"
  4. Ranking and sorting
     - Helpful reviews first
  5. Alert system
     - Negative spike detection

Business value:
  - Product improvement insights
  - Customer satisfaction tracking
  - Competitive analysis
  - Return prediction
  - Personalized recommendations

Example pipeline:
  1. Ingest reviews (API, scraper)
  2. Preprocess text
  3. Sentiment classification
  4. Aspect extraction
  5. Aggregate insights
  6. Dashboard and alerts
""")

# 3. FINANCIAL SENTIMENT
print("\n=== Financial Sentiment ===")
print("""
Applications:
  - Stock price prediction
  - Trading signals
  - Risk assessment
  - Earnings call analysis
  - News impact
  - Social trading sentiment

Data sources:
  - Financial news (Bloomberg, Reuters)
  - Earnings call transcripts
  - SEC filings
  - Twitter cashtags ($AAPL)
  - Reddit (r/wallstreetbets)
  - Analyst reports

Specific challenges:
  - Domain-specific language
  - Nuanced sentiment (hedging)
  - Market moving news
  - High stakes
  - Regulatory compliance
  - Speed (milliseconds matter)

Domain-specific models:
  - FinBERT (ProsusAI)
  - Financial PhraseBank
  - FiQA
  - Custom fine-tuned

Example metrics:
  - Daily sentiment score per stock
  - Correlation with returns
  - News impact analysis
  - Sentiment momentum
""")

# 4. CUSTOMER SERVICE
print("\n=== Customer Service ===")
print("""
Applications:
  - Ticket classification
  - Priority routing
  - Satisfaction prediction
  - Agent performance
  - Chat analysis
  - Escalation triggers

Tasks:
  1. Sentiment detection
     - Positive, negative, urgent
  2. Intent classification
     - Complaint, question, feedback
  3. Priority scoring
     - Route urgent first
  4. Auto-responses
     - Match to FAQ
  5. Agent assist
     - Real-time suggestions

Business value:
  - Faster response times
  - Better routing
  - Reduced churn
  - Improved satisfaction
  - Agent training insights

Example system:
  1. Incoming message (email, chat, call)
  2. Sentiment analysis
  3. Priority scoring
  4. Route to appropriate agent
  5. Agent dashboard with context
  6. Track resolution sentiment
""")

# 5. HEALTHCARE
print("\n=== Healthcare ===")
print("""
Applications:
  - Patient feedback analysis
  - Mental health monitoring
  - Drug review analysis
  - Clinical trial feedback
  - Physician reviews
  - Symptom tracking

Specific concerns:
  - Privacy (HIPAA, GDPR)
  - Sensitivity (mental health)
  - Accuracy critical
  - Bias concerns
  - Regulatory requirements

Use cases:
  1. Patient satisfaction
     - Survey responses
     - Google reviews
     - Social media
  
  2. Mental health
     - Depression detection
     - Suicide risk
     - Crisis intervention
  
  3. Pharma
     - Drug reviews
     - Side effect reports
     - Adherence

Ethical considerations:
  - Informed consent
  - Data anonymization
  - Human oversight
  - Clear boundaries
  - Do no harm
""")

# 6. POLITICS AND POLICY
print("\n=== Politics and Policy ===")
print("""
Applications:
  - Voter sentiment
  - Policy reaction
  - Debate analysis
  - Campaign tracking
  - News bias detection
  - Public opinion

Data sources:
  - Twitter/X
  - Facebook
  - Reddit
  - News comments
  - Surveys

Challenges:
  - Bot manipulation
  - Echo chambers
  - Polarization
  - Coordinated campaigns
  - Misinformation
  - Ethical use

Use cases:
  1. Campaign monitoring
     - Real-time sentiment
     - Topic tracking
     - Voter segments
  
  2. Policy analysis
     - Public reaction
     - Regional differences
     - Temporal trends
  
  3. Media analysis
     - Framing detection
     - Bias measurement
     - Coverage sentiment
""")

# 7. PRODUCT ANALYTICS
print("\n=== Product Analytics ===")
print("""
Applications:
  - App store reviews
  - User feedback
  - Feature requests
  - Bug reports
  - Usability issues
  - Competitive analysis

Pipeline:
  1. Aggregate reviews (App Store, Google Play)
  2. Sentiment classification
  3. Feature extraction
  4. Trend analysis
  5. Alert on issues
  6. Report to product team

Metrics:
  - Sentiment trend over versions
  - Feature-specific sentiment
  - Bug report frequency
  - User satisfaction score
  - Competitive comparison

Business impact:
  - Prioritize features
  - Fix bugs faster
  - Improve retention
  - Guide roadmap
""")

# 8. PRODUCTION SYSTEM
print("\n=== Production System Architecture ===")
print("""
Components:

1. DATA INGESTION
   - APIs (Twitter, Reddit, etc.)
   - Database queries
   - File uploads
   - Real-time streams

2. PREPROCESSING
   - Text cleaning
   - Language detection
   - Tokenization
   - Feature extraction

3. MODEL INFERENCE
   - Sentiment classification
   - Aspect extraction
   - Emotion detection
   - Sarcasm detection

4. POST-PROCESSING
   - Aggregation
   - Confidence filtering
   - Alert triggering
   - Storage

5. SERVING
   - REST API
   - GraphQL
   - WebSocket (real-time)
   - Batch processing

6. MONITORING
   - Accuracy tracking
   - Drift detection
   - Performance metrics
   - Error logs

7. FEEDBACK LOOP
   - User corrections
   - Active learning
   - Model retraining
   - Continuous improvement

Tech Stack Example:
  - Ingestion: Kafka, Airflow
  - Storage: PostgreSQL, S3
  - ML: PyTorch, Transformers
  - Serving: FastAPI, TorchServe
  - Monitoring: Grafana, Prometheus
""")

# 9. CASE STUDY: TWITTER BRAND MONITORING
print("\n=== Case Study: Twitter Brand Monitoring ===")
print("""
Company: Major airline
Goal: Monitor brand sentiment in real-time

Data:
  - 100,000 tweets/day mentioning brand
  - 30+ languages
  - Real-time requirements (< 1 minute)

Pipeline:
  1. Stream tweets via Twitter API
  2. Language detection (fastText)
  3. Route to language-specific model
  4. Sentiment classification (5 classes)
  5. Aspect extraction (service, delays, staff)
  6. Location extraction (for regional analysis)
  7. Alert on negative spikes

Models:
  - English: RoBERTa fine-tuned
  - Other languages: XLM-RoBERTa
  - Aspect: Custom BERT
  - Sarcasm: DistilBERT

Infrastructure:
  - Kafka for streaming
  - Ray for distributed inference
  - Redis for caching
  - Grafana dashboards

Results:
  - 85% accuracy across languages
  - Sub-minute latency
  - Real-time crisis detection
  - 40% faster response to issues
  - Improved brand perception

Key learnings:
  - Sarcasm is hard (20% of negative tweets)
  - Emojis are crucial (high signal)
  - Context matters (user history)
  - Regional differences significant
  - Continuous retraining needed
""")

🚀 Future: LLMs & Multimodal

🎯 The Next Generation

Sentiment analysis is being transformed by LLMs and multimodal models. GPT-4, Claude, and Gemini can analyze sentiment with explanations, handle complex cases, and understand multiple modalities.

Future Directions

LLMs: Zero-shot with explanations. Multimodal: Text + image + audio. Explainable: Reasoning behind predictions. Conversational: Interactive analysis. Personalized: User-specific models. Real-time: On-device inference.

🤖 GPT-4🎭 Multimodal💡 Explainable🎯 Personalized

# Future of sentiment analysis
import torch

# 1. LLM-BASED SENTIMENT (Zero-shot)
print("=== LLM-Based Sentiment ===")
print("""
Using GPT-4, Claude, or Gemini for sentiment:

Advantages:
  ✓ No training needed
  ✓ Zero-shot or few-shot
  ✓ Handles complex cases
  ✓ Provides explanations
  ✓ Multi-lingual
  ✓ Can ask for confidence

Example prompts:

1. Basic:
   "Analyze the sentiment of this text: {text}
    Return: positive, negative, or neutral."

2. With explanation:
   "Analyze the sentiment of this text.
    Provide:
    - Overall sentiment
    - Key phrases
    - Confidence (0-1)
    - Reasoning
    Text: {text}"

3. Aspect-based:
   "Analyze sentiment toward each aspect mentioned:
    Text: {text}
    Return JSON with aspects and their sentiments."

4. Few-shot:
   "Examples:
    Text: 'Great product!' → positive
    Text: 'Terrible service.' → negative
    Text: 'It was okay.' → neutral
    
    Text: '{new_text}' → "

Models to use:
  - GPT-4 / GPT-4 Turbo
  - Claude 3 (Opus, Sonnet, Haiku)
  - Gemini 1.5 Pro
  - Llama 3 (open source)
  - Mistral Large

When to use:
  - Complex/ambiguous texts
  - Need explanations
  - Multi-lingual
  - Low volume
  - No training data

When NOT to use:
  - High volume (expensive)
  - Low latency required
  - Simple texts
  - Privacy concerns
  - On-device needed
""")

# 2. MULTIMODAL SENTIMENT
print("\n=== Multimodal Sentiment ===")
print("""
Beyond text: combine multiple modalities.

Modalities:
  - Text (captions, comments)
  - Images (photos, memes)
  - Audio (tone, music)
  - Video (facial expressions)
  - Metadata (time, location, user)

Use cases:

1. MEME ANALYSIS
   - Text + image
   - Sarcasm detection
   - Cultural context
   - Hardest task

2. VIDEO REVIEWS
   - Audio sentiment
   - Facial expressions
   - Body language
   - Text captions

3. PRODUCT IMAGES
   - Visual quality
   - Defect detection
   - Aesthetic sentiment

4. SOCIAL MEDIA POSTS
   - Text + emoji + image
   - Combined sentiment
   - Context from metadata

Models:
  - CLIP (image + text)
  - Flamingo (DeepMind)
  - GPT-4V (vision)
  - Gemini (multimodal)
  - LLaVA (open source)

Challenges:
  - Modal alignment
  - Missing modalities
  - Computational cost
  - Data requirements
""")

# 3. EXPLAINABLE SENTIMENT
print("\n=== Explainable Sentiment ===")
print("""
Modern systems must explain their predictions.

Levels of explanation:

1. WORD-LEVEL
   - Highlight important words
   - Attention weights
   - SHAP values
   - Integrated gradients

2. PHRASE-LEVEL
   - Key phrases extraction
   - Aspect identification
   - Opinion targets

3. SENTENCE-LEVEL
   - Reasoning chains
   - Chain-of-thought
   - Natural language explanation

4. COUNTERFACTUAL
   - "If you changed X, sentiment would be Y"
   - Actionable insights
   - User understanding

Example output:
  Text: "The food was amazing but service was slow."
  
  Analysis:
    - Sentiment: Mixed (slightly negative)
    - Aspects:
      * Food: positive (amazing)
      * Service: negative (slow)
    - Reasoning:
      * "amazing" indicates strong positive
      * "but" signals contrast
      * "slow" indicates negative
    - Overall: Service issue outweighs food quality
    - Confidence: 0.85

Benefits:
  - Trust building
  - Debugging
  - User engagement
  - Compliance (GDPR Article 22)
  - Improvement insights
""")

# 4. PERSONALIZED SENTIMENT
print("\n=== Personalized Sentiment ===")
print("""
Adapting to individual users or contexts.

Personalization dimensions:

1. USER-SPECIFIC
   - Individual writing style
   - Slang patterns
   - Historical preferences
   - Cultural background

2. DOMAIN-SPECIFIC
   - Industry jargon
   - Technical terms
   - Domain conventions

3. CONTEXT-SPECIFIC
   - Current situation
   - Previous messages
   - Relationship history

4. CULTURAL
   - Regional expressions
   - Cultural norms
   - Language variants

Approach:
  1. Base model (general)
  2. User embedding
  3. Domain adaptation
  4. Context injection
  5. Continuous learning

Benefits:
  - Higher accuracy
  - Better user experience
  - Reduced errors
  - Trust

Challenges:
  - Privacy
  - Data requirements
  - Cold start
  - Complexity
""")

# 5. ON-DEVICE SENTIMENT
print("\n=== On-Device Sentiment ===")
print("""
Running sentiment analysis on user devices.

Benefits:
  ✓ Privacy (no data leaves device)
  ✓ Low latency (< 100ms)
  ✓ Offline capable
  ✓ No API costs
  ✓ Always available

Techniques:

1. MODEL COMPRESSION
   - Distillation (DistilBERT)
   - Pruning (remove weights)
   - Quantization (INT8/INT4)
   - Knowledge distillation

2. EFFICIENT ARCHITECTURES
   - MobileBERT
   - TinyBERT
   - ALBERT
   - EfficientFormer

3. HARDWARE OPTIMIZATION
   - ONNX Runtime
   - TensorFlow Lite
   - Core ML (Apple)
   - NNAPI (Android)

4. FRAMEWORKS
   - TFLite
   - PyTorch Mobile
   - Core ML
   - ONNX Runtime Mobile

Model sizes:
  - Full BERT: 440MB
  - DistilBERT: 250MB
  - MobileBERT: 100MB
  - TinyBERT: 55MB
  - Quantized TinyBERT: 15MB

Use cases:
  - Mobile apps
  - Smartwatches
  - IoT devices
  - Privacy-focused apps
  - Offline scenarios
""")

# 6. EMOTION DETECTION EVOLUTION
print("\n=== Emotion Detection Evolution ===")
print("""
From binary to fine-grained:

Evolution:
  1. Binary (positive/negative)
  2. Ternary (pos/neg/neutral)
  3. Fine-grained (5-star)
  4. Basic emotions (6 Ekman)
  5. Complex emotions (27+ GoEmotions)
  6. Continuous emotion (valence, arousal)
  7. Multi-dimensional (joy + surprise)

Emotion taxonomies:

Ekman's 6:
  - Happiness
  - Sadness
  - Anger
  - Fear
  - Surprise
  - Disgust

Plutchik's 8:
  - Joy, Trust, Fear, Surprise,
    Sadness, Disgust, Anger, Anticipation

GoEmotions (27):
  - Admiration, Amusement, Anger,
    Annoyance, Approval, Caring,
    Confusion, Curiosity, Desire,
    Disappointment, Disapproval, Disgust,
    Embarrassment, Excitement, Fear,
    Gratitude, Grief, Joy, Love,
    Nervousness, Optimism, Pride,
    Realization, Relief, Remorse,
    Sadness, Surprise

Applications:
  - Mental health
  - Customer experience
  - Content moderation
  - Personalized content
  - Social media analysis
""")

# 7. FUTURE DIRECTIONS
print("\n=== Future Directions ===")
print("""
1. UNIFIED MODELS
   - One model for all tasks
   - Sentiment + emotion + aspect
   - Multi-lingual
   - Multi-modal
   - Example: Gemini, GPT-4V

2. EXPLAINABLE BY DEFAULT
   - Every prediction explained
   - Natural language reasoning
   - Actionable insights
   - Regulatory compliance

3. CONTINUOUS LEARNING
   - Adapt to new slang
   - Learn from feedback
   - No retraining needed
   - Online learning

4. CAUSAL SENTIMENT
   - Why sentiment changed
   - Counterfactual analysis
   - Actionable insights
   - Root cause analysis

5. SENTIMENT GENERATION
   - Generate text with target sentiment
   - Style transfer
   - Personalization
   - Marketing content

6. ADVERSARIAL ROBUSTNESS
   - Detect manipulation
   - Fake reviews
   - Bot detection
   - Robust to attacks

7. PRIVACY-PRESERVING
   - Federated learning
   - Differential privacy
   - On-device processing
   - Homomorphic encryption
""")

# 8. OPEN CHALLENGES
print("\n=== Open Challenges ===")
print("""
1. SARCASM
   - Still unsolved
   - ~85% accuracy max
   - Cultural dependency
   - Context critical

2. IMPLICIT SENTIMENT
   - Not explicitly stated
   - "I've been waiting 3 hours"
   - Implies negative

3. MIXED SENTIMENT
   - "Good food, bad service"
   - Overall?
   - Aspect-specific

4. EVOLVING LANGUAGE
   - New slang
   - Memes
   - Cultural shifts
   - Continuous updates

5. LOW-RESOURCE LANGUAGES
   - Many languages underserved
   - Few datasets
   - Few models
   - Cultural context

6. INTERPRETABILITY
   - Deep models opaque
   - Need explanations
   - Regulatory requirements

7. DATA PRIVACY
   - Personal text sensitive
   - Federated learning
   - On-device
   - Anonymization
""")

# 9. RECOMMENDATIONS
print("\n=== Recommendations ===")
print("""
For practitioners:
  1. Start with pretrained BERT
  2. Fine-tune on your data
  3. Use LLMs for complex cases
  4. Consider multimodal
  5. Build explainability in
  6. Monitor continuously
  7. Plan for on-device

For researchers:
  1. Focus on sarcasm
  2. Multimodal fusion
  3. Low-resource languages
  4. Explainable AI
  5. Robustness
  6. Causal analysis
  7. Ethical considerations

For businesses:
  1. Start with off-the-shelf
  2. Build domain-specific model
  3. Integrate with workflows
  4. Track ROI
  5. Plan for scale
  6. Consider privacy
  7. Build feedback loops

For learners:
  1. Master fundamentals
  2. Understand transformers
  3. Practice on real data
  4. Build end-to-end project
  5. Learn deployment
  6. Stay updated
""")

# 10. FINAL THOUGHTS
print("\n=== Final Thoughts ===")
print("""
Sentiment analysis has evolved dramatically:

1980s-2000s: Rule-based
  - Lexicons
  - Hand-crafted rules
  - Limited scope

2010s: Machine learning
  - TF-IDF + classifiers
  - Word embeddings
  - Deep learning (LSTM, CNN)

2020s: Transformers + LLMs
  - BERT and variants
  - Zero-shot with GPT-4
  - Multimodal
  - Explainable

Future:
  - Unified foundation models
  - Perfect multi-lingual
  - Multimodal by default
  - Explainable always
  - Continuous learning
  - Privacy-preserving
  - On-device standard

Key trends:
  1. Bigger models → Better accuracy
  2. Multimodal → Richer understanding
  3. Explainable → Trust + compliance
  4. On-device → Privacy + speed
  5. Continuous → Always current

The goal:
  Understand human emotion
  At scale
  In any language
  With any modality
  Accurately
  Fairly
  Ethically

You now understand:
- Fundamentals of sentiment
- All major approaches
- Classical to modern
- Evaluation methods
- Production systems
- Future directions

"Sentiment analysis is the bridge
between human emotion and
machine understanding."
  — Anonymous

Go build systems that understand
what people really mean!
""")