🧠 Fundamentals

🎯 What is MongoDB?

MongoDB is a distributed document database designed for scale-out and high availability. It stores data as flexible JSON-like documents, supports horizontal scaling via sharding, and is a popular choice for Big Data applications that need fast writes and flexible schemas.

Why MongoDB for Big Data

Flexible: JSON documents. Scalable: Sharding. Fast: In-memory + indexes. Available: Replica sets. Aggregation: Pipeline. Integration: Spark, Kafka.

📌 Document Store📌 Sharding📌 Replica Sets📌 Aggregation
# MongoDB for Big Data — the big picture # Distributed document database for scale-out applications. # CONCEPTUAL OVERVIEW print(""" === MongoDB for Big Data === What it is: - Document database - JSON-like documents - Distributed - Scalable - Flexible schema Key features: - Documents and collections - Sharding (horizontal scale) - Replica sets (HA) - Aggregation pipeline - Indexes - Spark/Kafka integration What it's good for: - High write throughput - Flexible schema - Real-time analytics - IoT, logs, catalogs - Operational data What it's NOT: - Not OLAP warehouse - Not for complex joins - Not for strong ACID across collections - Not for analytical queries """) # SIMPLE EXAMPLE print("=== Simple Example ===") print(""" // Insert document db.users.insertOne({ _id: ObjectId(), name: "Alice", age: 30, city: "NYC", tags: ["python", "spark"], created_at: new Date() }); // Query db.users.find({ city: "NYC", age: { $gt: 25 } }); // Aggregation db.users.aggregate([ { $match: { city: "NYC" } }, { $group: { _id: "$city", count: { $sum: 1 } } } ]); """) # KEY PROPERTIES print(""" === Key Properties === 1. DOCUMENT MODEL - JSON-like documents - Nested structures - Arrays - Flexible schema 2. DISTRIBUTED - Sharding - Horizontal scale - Petabytes 3. HIGH AVAILABILITY - Replica sets - Automatic failover - Multi-region 4. AGGREGATION - Pipeline - Stages - Analytics 5. INDEXES - B-tree - Compound - Text - Geospatial 6. INTEGRATION - Spark - Kafka - BI tools """) # REAL-WORLD print("=== Real-World Scale ===") print(""" MongoDB: - Used by thousands of companies - Petabytes of data - Millions of operations/sec Examples: - Facebook (messages) - eBay (product catalog) - Adobe (creative cloud) - Bosch (IoT) - Coinbase (crypto) """)

🏗️ Architecture

🎯 Distributed Architecture

MongoDB architecture has three main components: mongos (routers), config servers (metadata), and shards (data). Replica sets provide high availability.

Architecture

Mongos: Query router. Config: Metadata. Shard: Data partition. Replica set: HA. Shard key: Partition key. Chunk: Data range.

🏗️ Mongos🏗️ Config Servers🏗️ Shards🏗️ Replica Sets
# MongoDB Architecture print("=== Architecture Diagram ===") print(""" ┌─────────────────────────────────────────┐ │ Client │ │ (App, BI, Spark) │ └─────────────────┬───────────────────────┘ │ ▼ ┌─────────────────────────────────────────┐ │ Mongos │ │ (Query Router) │ │ │ │ - Routes queries │ │ - Merges results │ │ - No data │ └─────────────────┬───────────────────────┘ │ ┌───────────┼───────────┐ │ │ │ ▼ ▼ ▼ ┌─────────┐ ┌─────────┐ ┌─────────┐ │Shard 1 │ │Shard 2 │ │Shard 3 │ │ │ │ │ │ │ │Replica │ │Replica │ │Replica │ │Set │ │Set │ │Set │ │ │ │ │ │ │ │Primary │ │Primary │ │Primary │ │Secondary│ │Secondary│ │Secondary│ │Secondary│ │Secondary│ │Secondary│ └─────────┘ └─────────┘ └─────────┘ │ │ │ └───────────┼───────────┘ │ ▼ ┌─────────────────────────────────────────┐ │ Config Servers │ │ (Metadata) │ │ │ │ - Cluster metadata │ │ - Chunk locations │ │ - Shard key ranges │ └─────────────────────────────────────────┘ """) print("\n=== Components ===") print(""" 1. MONGOS - Query router - Routes to shards - Merges results - Stateless - Multiple for HA 2. CONFIG SERVERS - Metadata store - Cluster state - Chunk locations - Replica set (3 nodes) 3. SHARDS - Data storage - Replica set per shard - Primary + secondaries - Horizontal scale 4. REPLICA SET - Primary (writes) - Secondaries (reads) - Automatic failover - 3+ nodes 5. SHARD KEY - Partition key - Determines distribution - Critical for performance 6. CHUNK - Range of shard key values - Migrated between shards - Balancer moves chunks """) print("\n=== Sharding ===") print(""" Sharding: - Horizontal partitioning - Data distributed across shards - Scales writes and storage Shard key: - Field or compound - Determines distribution - Must be chosen carefully Example: sh.shardCollection("db.users", { "user_id": "hashed" }) sh.shardCollection("db.orders", { "customer_id": 1, "date": 1 }) """) # IMPLEMENTATION print("\n=== Simulating MongoDB Architecture ===") class MongoShard: """Simulate a MongoDB shard.""" def __init__(self, shard_id): self.id = shard_id self.data = {} # collection -> [documents] self.state = 'active' def insert(self, collection, document): """Insert document.""" if collection not in self.data: self.data[collection] = [] self.data[collection].append(document) return len(self.data[collection]) - 1 def find(self, collection, query=None): """Find documents.""" docs = self.data.get(collection, []) if not query: return docs return [d for d in docs if all(d.get(k) == v for k, v in query.items())] def count(self, collection): """Count documents.""" return len(self.data.get(collection, [])) class MongoShardedCluster: """Simulate MongoDB sharded cluster.""" def __init__(self): self.shards = {} self.config = {} # collection -> shard key self.chunks = {} # collection -> [chunk ranges] def add_shard(self, shard_id): """Add shard.""" self.shards[shard_id] = MongoShard(shard_id) print(f"Cluster: Added shard {shard_id}") def shard_collection(self, collection, shard_key): """Shard collection.""" self.config[collection] = shard_key self.chunks[collection] = [] print(f"Cluster: Sharded '{collection}' by {shard_key}") def _get_shard_for_key(self, collection, document): """Determine shard for document.""" shard_key = self.config.get(collection) if not shard_key: # No sharding — use first shard return list(self.shards.keys())[0] # Simple hash-based key_value = document.get(shard_key) shard_ids = list(self.shards.keys()) index = hash(str(key_value)) % len(shard_ids) return shard_ids[index] def insert(self, collection, document): """Insert document.""" shard_id = self._get_shard_for_key(collection, document) shard = self.shards[shard_id] offset = shard.insert(collection, document) return shard_id, offset def find(self, collection, query=None): """Find documents across shards.""" results = [] for shard in self.shards.values(): results.extend(shard.find(collection, query)) return results def count(self, collection): """Count across shards.""" return sum(s.count(collection) for s in self.shards.values()) def get_stats(self): """Get cluster stats.""" return { 'shards': len(self.shards), 'collections': len(self.config), 'documents': sum( s.count(c) for s in self.shards.values() for c in ['users', 'orders'] ), } # Demo print("=== Sharded Cluster Demo ===") cluster = MongoShardedCluster() # Add shards for i in range(1, 4): cluster.add_shard(f"shard_{i}") # Shard collection cluster.shard_collection('users', 'user_id') # Insert documents print("\nInserting users:") for i in range(10): doc = {'user_id': i, 'name': f'user_{i}', 'age': 20 + i} shard_id, offset = cluster.insert('users', doc) print(f" user_{i} → {shard_id}") # Find print("\nFinding users:") results = cluster.find('users', {'age': 25}) for doc in results: print(f" {doc}") # Count print(f"\nTotal users: {cluster.count('users')}") print(f"Cluster stats: {cluster.get_stats()}") # ARCHITECTURE SUMMARY print("\n=== Architecture Summary ===") print(f"{'Component':<25} {'Role':<40}") print("-" * 65) print(f"{'Mongos':<25} {'Query router':<40}") print(f"{'Config Servers':<25} {'Metadata':<40}") print(f"{'Shards':<25} {'Data storage':<40}") print(f"{'Replica Set':<25} {'HA':<40}") print(f"{'Shard Key':<25} {'Partition key':<40}") print(f"{'Chunk':<25} {'Data range':<40}")

📄 Documents & Collections

🎯 The Document Model

MongoDB stores data as documents (JSON-like BSON) in collections. Documents are flexible, can have nested structures, and don't require a fixed schema.

Documents & Collections

Document: JSON-like. Collection: Group of documents. BSON: Binary JSON. Schema: Flexible. Nested: Subdocuments. Arrays: Lists.

📄 Document📄 Collection📄 BSON📄 Flexible Schema
# Documents & Collections print("=== Document Model ===") print(""" Document: - JSON-like - BSON (binary JSON) - Key-value pairs - Nested - Arrays Example: { _id: ObjectId("..."), name: "Alice", age: 30, address: { city: "NYC", zip: "10001" }, tags: ["python", "spark"], created_at: ISODate("2024-01-01") } Collection: - Group of documents - Like a table (but flexible) - No fixed schema """) print("\n=== Data Types ===") print(""" BSON types: - String - Integer (32/64) - Double - Boolean - Date - ObjectId - Array - Object (nested) - Null - Binary - Decimal128 - Timestamp """) print("\n=== Operations ===") print(""" // Insert db.users.insertOne({ name: "Alice", age: 30 }); db.users.insertMany([...]); // Find db.users.find({ age: { $gt: 25 } }); db.users.findOne({ name: "Alice" }); // Update db.users.updateOne( { name: "Alice" }, { $set: { age: 31 } } ); // Delete db.users.deleteOne({ name: "Alice" }); // Aggregate db.users.aggregate([ { $match: { city: "NYC" } }, { $group: { _id: "$city", count: { $sum: 1 } } } ]); """) print("\n=== Nested Documents ===") print(""" // Nested document { user: { name: "Alice", address: { city: "NYC", zip: "10001" } } } // Query nested db.users.find({ "user.address.city": "NYC" }); // Array { name: "Alice", tags: ["python", "spark", "sql"] } // Query array db.users.find({ tags: "python" }); db.users.find({ tags: { $all: ["python", "spark"] } }); """) print("\n=== When to Use ===") print(""" USE DOCUMENTS WHEN: ✓ Flexible schema ✓ Nested data ✓ Rapid development ✓ JSON APIs ✓ Content management ✓ IoT, logs, catalogs AVOID WHEN: ✗ Complex joins needed ✗ Multi-table ACID ✗ Heavy analytics """)

🔀 Sharding

🎯 Horizontal Scaling

Sharding distributes data across multiple servers (shards). Each shard holds a subset of the data, enabling horizontal scaling of writes and storage.

Sharding

Shard key: Partition key. Chunks: Data ranges. Balancer: Migrates chunks. Hashed: Even distribution. Ranged: Range queries. Compound: Multiple fields.

🔀 Shard Key🔀 Chunks🔀 Balancer🔀 Hashed
# Sharding print("=== What is Sharding? ===") print(""" Sharding: - Horizontal partitioning - Data across shards - Scales writes and storage Without sharding: - Single server - Limited capacity With sharding: - Multiple servers - Unlimited capacity - Parallel writes """) print("\n=== Shard Key ===") print(""" Shard key: - Field or compound - Determines distribution - Immutable (mostly) - Critical for performance Types: - Hashed: Even distribution - Ranged: Range queries - Compound: Multiple fields Examples: sh.shardCollection("db.users", { "user_id": "hashed" }) sh.shardCollection("db.orders", { "customer_id": 1, "date": 1 }) """) print("\n=== Chunks ===") print(""" Chunks: - Range of shard key values - Default: 64 MB - Migrated by balancer - Split when large Example: Chunk 1: user_id 0-1000 Chunk 2: user_id 1000-2000 Chunk 3: user_id 2000-3000 ... """) print("\n=== Choosing a Shard Key ===") print(""" Good shard key: - High cardinality - Even distribution - Common query field - Immutable Bad shard key: - Low cardinality - Monotonic (timestamp) - Rarely queried - Changing Example: BAD: { created_at: 1 } // Monotonic! GOOD: { user_id: "hashed" } // Even GOOD: { customer_id: 1, date: 1 } // Compound """) print("\n=== Hashed vs Ranged ===") print(f"{'Aspect':<25} {'Hashed':<25} {'Ranged':<25}") print("-" * 75) print(f"{'Distribution':<25} {'Even':<25} {'Range-based':<25}") print(f"{'Range queries':<25} {'Scatter-gather':<25} {'Efficient':<25}") print(f"{'Monotonic':<25} {'OK':<25} {'Hotspot':<25}") print(f"{'Use case':<25} {'Even writes':<25} {'Range queries':<25}") # IMPLEMENTATION print("\n=== Simulating Sharding ===") class ShardedCollection: """Simulate sharded collection.""" def __init__(self, name, shard_key, num_shards=3): self.name = name self.shard_key = shard_key self.num_shards = num_shards self.shards = [[] for _ in range(num_shards)] self.chunks = [] def _get_shard(self, document): """Get shard for document.""" key_value = document.get(self.shard_key) # Hashed return hash(str(key_value)) % self.num_shards def insert(self, document): """Insert document.""" shard = self._get_shard(document) self.shards[shard].append(document) return shard def get_distribution(self): """Get shard distribution.""" return [len(s) for s in self.shards] def query(self, key_value): """Query by shard key.""" shard = hash(str(key_value)) % self.num_shards return [d for d in self.shards[shard] if d.get(self.shard_key) == key_value] def scatter_gather(self, predicate): """Query all shards.""" results = [] for shard in self.shards: results.extend([d for d in shard if predicate(d)]) return results # Demo print("=== Sharding Demo ===") # Create sharded collection users = ShardedCollection('users', 'user_id', num_shards=3) # Insert documents print("Inserting 20 users:") for i in range(20): shard = users.insert({'user_id': i, 'name': f'user_{i}'}) # Distribution print(f"\nShard distribution: {users.get_distribution()}") # Query by shard key print("\nQuery user_id=5:") result = users.query(5) print(f" {result}") # Scatter-gather print("\nScatter-gather (name contains '1'):") result = users.scatter_gather(lambda d: '1' in d['name']) print(f" Found {len(result)} documents") # SHARDING SUMMARY print("\n=== Sharding Summary ===") print(f"{'Concept':<25} {'Description':<40}") print("-" * 65) print(f"{'Shard key':<25} {'Partition key':<40}") print(f"{'Chunks':<25} {'Data ranges':<40}") print(f"{'Balancer':<25} {'Migrates chunks':<40}") print(f"{'Hashed':<25} {'Even distribution':<40}") print(f"{'Ranged':<25} {'Range queries':<40}")

🔄 Replication

🎯 High Availability

Replica sets provide high availability. One primary handles writes; secondaries replicate data and can serve reads. Automatic failover ensures availability.

Replication

Primary: Writes. Secondary: Replicates. Arbiter: Votes. Failover: Automatic. Read preference: Control. Oplog: Replication log.

🔄 Primary🔄 Secondary🔄 Failover🔄 Oplog
# Replication print("=== What is a Replica Set? ===") print(""" Replica Set: - Group of MongoDB nodes - One primary - Multiple secondaries - Automatic failover - High availability Nodes: - Primary: Handles writes - Secondary: Replicates - Arbiter: Votes (no data) """) print("\n=== Replication Flow ===") print(""" 1. Client writes to primary 2. Primary writes to oplog 3. Secondaries read oplog 4. Secondaries apply changes 5. Secondaries acknowledge 6. Primary acknowledges client """) print("\n=== Failover ===") print(""" Automatic failover: 1. Primary fails 2. Secondaries detect 3. Election 4. New primary 5. Clients reconnect 6. Resume Time: Seconds Downtime: Minimal """) print("\n=== Read Preference ===") print(""" Modes: - primary: Only primary - primaryPreferred: Primary if available - secondary: Only secondary - secondaryPreferred: Secondary if available - nearest: Nearest node Example: db.collection.find().readPref("secondary") """) print("\n=== Write Concern ===") print(""" Write concern: - w: 1 (primary) - w: majority - w: 0 (fire and forget) - j: journal Example: db.collection.insertOne(doc, { writeConcern: { w: "majority" } }) """) print("\n=== Configuration ===") print(""" # Start replica set mongod --replSet rs0 --port 27017 mongod --replSet rs0 --port 27018 mongod --replSet rs0 --port 27019 # Initiate rs.initiate({ _id: "rs0", members: [ { _id: 0, host: "localhost:27017" }, { _id: 1, host: "localhost:27018" }, { _id: 2, host: "localhost:27019" } ] }); # Check status rs.status(); """) # IMPLEMENTATION print("\n=== Simulating Replication ===") class ReplicaSet: """Simulate MongoDB replica set.""" def __init__(self, name): self.name = name self.nodes = [] self.primary = None def add_node(self, node_id, is_primary=False): """Add node.""" node = { 'id': node_id, 'data': [], 'state': 'primary' if is_primary else 'secondary', 'oplog': [], } self.nodes.append(node) if is_primary: self.primary = node print(f"ReplicaSet: Added {node_id} ({node['state']})") def write(self, document): """Write to primary.""" if not self.primary: return False self.primary['data'].append(document) self.primary['oplog'].append({'op': 'insert', 'doc': document}) # Replicate to secondaries for node in self.nodes: if node != self.primary: node['data'].append(document) node['oplog'].append({'op': 'insert', 'doc': document}) return True def failover(self): """Simulate primary failure.""" if not self.primary: return print(f"ReplicaSet: Primary {self.primary['id']} failed") self.primary['state'] = 'down' # Elect new primary for node in self.nodes: if node['state'] == 'secondary': node['state'] = 'primary' self.primary = node print(f"ReplicaSet: New primary {node['id']}") return self.primary = None def get_stats(self): """Get replica set stats.""" return { 'name': self.name, 'nodes': len(self.nodes), 'primary': self.primary['id'] if self.primary else None, 'data_count': len(self.primary['data']) if self.primary else 0, } # Demo print("=== Replication Demo ===") rs = ReplicaSet('rs0') # Add nodes rs.add_node('node1', is_primary=True) rs.add_node('node2') rs.add_node('node3') # Write print("\nWriting documents:") for i in range(5): rs.write({'id': i, 'name': f'doc_{i}'}) print(f" Wrote doc_{i}") print(f"\nStats: {rs.get_stats()}") # Failover print("\nSimulating failover:") rs.failover() print(f"\nStats after failover: {rs.get_stats()}") # REPLICATION SUMMARY print("\n=== Replication Summary ===") print(f"{'Concept':<25} {'Description':<40}") print("-" * 65) print(f"{'Primary':<25} {'Writes':<40}") print(f"{'Secondary':<25} {'Replicates':<40}") print(f"{'Failover':<25} {'Automatic':<40}") print(f"{'Oplog':<25} {'Replication log':<40}") print(f"{'Read preference':<25} {'Control reads':<40}")

📊 Aggregation Pipeline

🎯 Analytics in MongoDB

The aggregation pipeline processes documents through stages: match, group, sort, project, and more. It's MongoDB's analytics engine.

Aggregation

Pipeline: Stages. $match: Filter. $group: Aggregate. $sort: Order. $project: Shape. $lookup: Join.

📊 Pipeline📊 Stages📊 $match📊 $group
# Aggregation Pipeline print("=== What is Aggregation? ===") print(""" Aggregation: - Pipeline of stages - Each stage transforms - Output of one → input of next - Analytics Stages: - $match: Filter - $group: Aggregate - $sort: Order - $project: Shape - $lookup: Join - $unwind: Flatten arrays - $limit: Limit - $skip: Skip """) print("\n=== Example Pipeline ===") print(""" db.orders.aggregate([ // Stage 1: Filter { $match: { status: "completed" } }, // Stage 2: Group { $group: { _id: "$customer_id", total: { $sum: "$amount" }, count: { $sum: 1 } }}, // Stage 3: Sort { $sort: { total: -1 } }, // Stage 4: Limit { $limit: 10 }, // Stage 5: Project { $project: { _id: 0, customer_id: "$_id", total: 1, count: 1 }} ]); """) print("\n=== Common Stages ===") print(""" $match: { $match: { age: { $gt: 30 } } } $group: { $group: { _id: "$city", count: { $sum: 1 } } } $sort: { $sort: { count: -1 } } $project: { $project: { name: 1, age: 1, _id: 0 } } $lookup: { $lookup: { from: "orders", localField: "_id", foreignField: "user_id", as: "orders" }} $unwind: { $unwind: "$tags" } $limit: { $limit: 10 } $skip: { $skip: 20 } """) print("\n=== Aggregation vs MapReduce ===") print(""" Aggregation: - Pipeline - Declarative - Fast - Recommended MapReduce: - JavaScript - Deprecated (mostly) - Slower - Use aggregation instead """) # IMPLEMENTATION print("\n=== Simulating Aggregation ===") class AggregationPipeline: """Simulate aggregation pipeline.""" def __init__(self, data): self.data = data def match(self, condition): """Filter.""" return AggregationPipeline([d for d in self.data if condition(d)]) def group(self, key_func, agg_funcs): """Group.""" groups = {} for doc in self.data: key = key_func(doc) if key not in groups: groups[key] = [] groups[key].append(doc) results = [] for key, docs in groups.items(): result = {'_id': key} for name, func in agg_funcs.items(): result[name] = func(docs) results.append(result) return AggregationPipeline(results) def sort(self, key, reverse=False): """Sort.""" return AggregationPipeline(sorted(self.data, key=lambda d: d.get(key, 0), reverse=reverse)) def limit(self, n): """Limit.""" return AggregationPipeline(self.data[:n]) def project(self, fields): """Project.""" results = [] for doc in self.data: result = {} for field in fields: if field in doc: result[field] = doc[field] results.append(result) return AggregationPipeline(results) def to_list(self): return self.data # Demo print("=== Aggregation Demo ===") # Sample data orders = [ {'order_id': 1, 'customer': 'Alice', 'amount': 100, 'status': 'completed'}, {'order_id': 2, 'customer': 'Bob', 'amount': 200, 'status': 'completed'}, {'order_id': 3, 'customer': 'Alice', 'amount': 150, 'status': 'completed'}, {'order_id': 4, 'customer': 'Charlie', 'amount': 300, 'status': 'pending'}, {'order_id': 5, 'customer': 'Bob', 'amount': 250, 'status': 'completed'}, {'order_id': 6, 'customer': 'Alice', 'amount': 50, 'status': 'completed'}, ] print("Original orders:") for o in orders: print(f" {o}") # Pipeline pipeline = AggregationPipeline(orders) result = pipeline \ .match(lambda d: d['status'] == 'completed') \ .group( key_func=lambda d: d['customer'], agg_funcs={ 'total': lambda docs: sum(d['amount'] for d in docs), 'count': len, } ) \ .sort('total', reverse=True) \ .to_list() print("\nAggregated (completed orders by customer):") for r in result: print(f" {r}") # AGGREGATION SUMMARY print("\n=== Aggregation Summary ===") print(f"{'Stage':<25} {'Description':<40}") print("-" * 65) print(f"{'$match':<25} {'Filter':<40}") print(f"{'$group':<25} {'Aggregate':<40}") print(f"{'$sort':<25} {'Order':<40}") print(f"{'$project':<25} {'Shape':<40}") print(f"{'$lookup':<25} {'Join':<40}") print(f"{'$unwind':<25} {'Flatten arrays':<40}")

🔍 Indexes

🎯 Fast Queries

Indexes speed up queries by avoiding full collection scans. MongoDB supports many index types: single field, compound, text, geospatial, and more.

Indexes

B-tree: Standard. Compound: Multiple fields. Text: Full-text. Geospatial: 2dsphere. TTL: Expiration. Unique: Constraint.

🔍 B-tree🔍 Compound🔍 Text🔍 Geospatial
# Indexes print("=== What is an Index? ===") print(""" Index: - Data structure - Speeds up queries - Avoids full scan - Trade-off: writes slower Without index: - Full collection scan - Slow for large collections With index: - Direct lookup - Fast """) print("\n=== Index Types ===") print(""" 1. SINGLE FIELD db.users.createIndex({ name: 1 }) 2. COMPOUND db.users.createIndex({ city: 1, age: -1 }) 3. UNIQUE db.users.createIndex({ email: 1 }, { unique: true }) 4. TEXT db.articles.createIndex({ content: "text" }) 5. GEOSPATIAL db.places.createIndex({ location: "2dsphere" }) 6. TTL db.sessions.createIndex({ created_at: 1 }, { expireAfterSeconds: 3600 }) 7. HASHED db.users.createIndex({ user_id: "hashed" }) """) print("\n=== Compound Indexes ===") print(""" Compound: - Multiple fields - Order matters - Prefix queries Example: db.users.createIndex({ city: 1, age: -1 }) Supports: ✓ { city: "NYC" } ✓ { city: "NYC", age: 30 } ✓ { city: "NYC", age: { $gt: 25 } } ✗ { age: 30 } // No prefix """) print("\n=== Index Best Practices ===") print(""" 1. Index query fields 2. Index sort fields 3. Use compound for multiple 4. Prefix matters 5. Avoid over-indexing 6. Monitor usage 7. Use explain() 8. Consider cardinality """) print("\n=== Explain ===") print(""" db.users.find({ city: "NYC" }).explain("executionStats") Output: - winningPlan - executionStats - index usage - docs examined - time """) # IMPLEMENTATION print("\n=== Simulating Indexes ===") class IndexedCollection: """Simulate indexed collection.""" def __init__(self): self.data = [] self.indexes = {} def insert(self, document): """Insert document.""" self.data.append(document) # Update indexes for field, index in self.indexes.items(): value = document.get(field) if value not in index: index[value] = [] index[value].append(document) def create_index(self, field): """Create index.""" index = {} for doc in self.data: value = doc.get(field) if value not in index: index[value] = [] index[value].append(doc) self.indexes[field] = index print(f"Index created on '{field}'") def find_with_index(self, field, value): """Find using index.""" if field in self.indexes: print(f" Using index on '{field}'") return self.indexes[field].get(value, []) else: print(f" No index — full scan") return [d for d in self.data if d.get(field) == value] def find_without_index(self, field, value): """Find without index.""" print(f" Full scan") return [d for d in self.data if d.get(field) == value] # Demo print("=== Index Demo ===") # Create collection users = IndexedCollection() # Insert documents for i in range(100): users.insert({ 'id': i, 'name': f'user_{i}', 'city': ['NYC', 'LA', 'Chicago'][i % 3], }) print(f"Inserted {len(users.data)} documents") # Without index print("\nWithout index:") result = users.find_without_index('city', 'NYC') print(f" Found {len(result)} documents") # Create index print("\nCreating index:") users.create_index('city') # With index print("\nWith index:") result = users.find_with_index('city', 'NYC') print(f" Found {len(result)} documents") # INDEX SUMMARY print("\n=== Index Summary ===") print(f"{'Type':<25} {'Use Case':<40}") print("-" * 65) print(f"{'Single field':<25} {'Equality, range':<40}") print(f"{'Compound':<25} {'Multiple fields':<40}") print(f"{'Unique':<25} {'Constraint':<40}") print(f"{'Text':<25} {'Full-text search':<40}") print(f"{'Geospatial':<25} {'Location queries':<40}") print(f"{'TTL':<25} {'Expiration':<40}")

⚡ MongoDB + Spark

🎯 Big Data Analytics

The MongoDB Spark Connector lets Spark read and write MongoDB data. This combines MongoDB's operational store with Spark's analytics power.

MongoDB + Spark

Connector: Read/write. DataFrames: Spark SQL. RDDs: Low-level. Streaming: Real-time. Aggregation: Push down. ML: Spark MLlib.

⚡ Connector⚡ DataFrame⚡ Streaming⚡ ML
# MongoDB + Spark print("=== MongoDB Spark Connector ===") print(""" Spark Connector: - Read from MongoDB - Write to MongoDB - DataFrame API - RDD API - Streaming Use cases: - Analytics on MongoDB data - ML on MongoDB data - ETL - Real-time """) print("\n=== Reading ===") print(""" # PySpark from pyspark.sql import SparkSession spark = SparkSession.builder \\ .appName("MongoDB") \\ .config("spark.mongodb.read.connection.uri", "mongodb://localhost:27017/analytics.users") \\ .config("spark.mongodb.write.connection.uri", "mongodb://localhost:27017/analytics.output") \\ .getOrCreate() # Read df = spark.read \\ .format("mongodb") \\ .option("connection.uri", "mongodb://localhost:27017") \\ .option("database", "analytics") \\ .option("collection", "users") \\ .load() df.show() # Query with aggregation df = spark.read \\ .format("mongodb") \\ .option("connection.uri", "mongodb://localhost:27017") \\ .option("database", "analytics") \\ .option("collection", "users") \\ .option("aggregation.pipeline", '[{"$match": {"age": {"$gt": 30}}}]') \\ .load() """) print("\n=== Writing ===") print(""" # Write df.write \\ .format("mongodb") \\ .mode("append") \\ .option("connection.uri", "mongodb://localhost:27017") \\ .option("database", "analytics") \\ .option("collection", "output") \\ .save() # Modes: # append: Add # overwrite: Replace # ignore: Skip if exists # error: Fail if exists """) print("\n=== Streaming ===") print(""" # Read stream stream = spark.readStream \\ .format("mongodb") \\ .option("connection.uri", "mongodb://localhost:27017") \\ .option("database", "analytics") \\ .option("collection", "events") \\ .load() # Process result = stream.groupBy("user_id").count() # Write stream query = result.writeStream \\ .format("mongodb") \\ .option("checkpointLocation", "/checkpoint") \\ .option("connection.uri", "mongodb://localhost:27017") \\ .option("database", "analytics") \\ .option("collection", "output") \\ .start() """) print("\n=== Benefits ===") print(""" 1. ANALYTICS - Spark SQL - DataFrames - ML 2. SCALE - Spark cluster - Parallel reads 3. INTEGRATION - MongoDB data - Spark ecosystem 4. REAL-TIME - Structured Streaming - Change streams """) # IMPLEMENTATION print("\n=== Simulating MongoDB + Spark ===") class MongoSparkConnector: """Simulate MongoDB Spark Connector.""" def __init__(self, mongo_data): self.mongo_data = mongo_data self.spark_data = [] def read(self, query=None): """Read from MongoDB.""" print(f"MongoDB Spark: Reading from MongoDB") data = self.mongo_data if query: data = [d for d in data if query(d)] self.spark_data = data print(f" Read {len(data)} documents") return data def transform(self, func): """Transform with Spark.""" print(f"MongoDB Spark: Transforming") self.spark_data = [func(d) for d in self.spark_data] return self.spark_data def write(self, collection): """Write to MongoDB.""" print(f"MongoDB Spark: Writing to {collection}") print(f" Wrote {len(self.spark_data)} documents") return len(self.spark_data) # Demo print("=== MongoDB + Spark Demo ===") # Sample MongoDB data mongo_data = [ {'user_id': 1, 'name': 'Alice', 'age': 30, 'city': 'NYC'}, {'user_id': 2, 'name': 'Bob', 'age': 25, 'city': 'LA'}, {'user_id': 3, 'name': 'Charlie', 'age': 35, 'city': 'NYC'}, {'user_id': 4, 'name': 'Diana', 'age': 28, 'city': 'LA'}, ] # Create connector connector = MongoSparkConnector(mongo_data) # Read data = connector.read(query=lambda d: d['age'] > 28) print(f" Data: {data}") # Transform transformed = connector.transform(lambda d: {**d, 'age_group': 'senior' if d['age'] > 30 else 'adult'}) print(f" Transformed: {transformed}") # Write count = connector.write('analytics.adults') print(f" Written: {count}") # SPARK SUMMARY print("\n=== MongoDB + Spark Summary ===") print(f"{'Feature':<25} {'Description':<40}") print("-" * 65) print(f"{'Read':<25} {'MongoDB → Spark':<40}") print(f"{'Write':<25} {'Spark → MongoDB':<40}") print(f"{'DataFrame':<25} {'Spark SQL':<40}") print(f"{'Streaming':<25} {'Structured Streaming':<40}") print(f"{'ML':<25} {'Spark MLlib':<40}")

📨 MongoDB + Kafka

🎯 Real-Time Pipelines

The MongoDB Kafka Connector streams data changes from MongoDB to Kafka and vice versa. This enables real-time data pipelines.

MongoDB + Kafka

Source: MongoDB → Kafka. Sink: Kafka → MongoDB. Change streams: Real-time. CDC: Change data capture. Exactly-once: Semantics.

📨 Source📨 Sink📨 Change Streams📨 CDC
# MongoDB + Kafka print("=== MongoDB Kafka Connector ===") print(""" Connector: - Source: MongoDB → Kafka - Sink: Kafka → MongoDB - Change streams - Real-time - Exactly-once Use cases: - Real-time pipelines - CDC - Event-driven - Data sync """) print("\n=== Source Connector ===") print(""" Source: - MongoDB changes → Kafka - Change streams - Insert, update, delete Configuration: { "name": "mongo-source", "config": { "connector.class": "com.mongodb.kafka.connect.MongoSourceConnector", "connection.uri": "mongodb://localhost:27017", "database": "analytics", "collection": "users", "topic.prefix": "mongo" } } """) print("\n=== Sink Connector ===") print(""" Sink: - Kafka → MongoDB - Insert, update, delete Configuration: { "name": "mongo-sink", "config": { "connector.class": "com.mongodb.kafka.connect.MongoSinkConnector", "connection.uri": "mongodb://localhost:27017", "database": "analytics", "collection": "output", "topics": "mongo.users" } } """) print("\n=== Change Streams ===") print(""" Change streams: - Watch collection changes - Insert, update, delete - Real-time - Resume tokens Example: const changeStream = db.users.watch(); changeStream.on('change', (change) => { console.log(change); }); """) print("\n=== Pipeline Example ===") print(""" MongoDB → Kafka → Spark → MongoDB 1. MongoDB source connector 2. Kafka topic 3. Spark Streaming 4. MongoDB sink connector Result: - Real-time analytics - Enriched data - Stored back """) # IMPLEMENTATION print("\n=== Simulating MongoDB + Kafka ===") class MongoKafkaPipeline: """Simulate MongoDB Kafka pipeline.""" def __init__(self): self.mongo_data = [] self.kafka_topics = {} self.processed = [] def insert_mongo(self, collection, document): """Insert into MongoDB.""" self.mongo_data.append({ 'collection': collection, 'document': document, 'operation': 'insert', }) print(f"MongoDB: Inserted into {collection}: {document}") def source_connector(self, collection, topic): """Simulate source connector.""" changes = [d for d in self.mongo_data if d['collection'] == collection] self.kafka_topics[topic] = changes print(f"Kafka: Source connector published {len(changes)} changes to {topic}") def process(self, topic, transform): """Process with Spark.""" changes = self.kafka_topics.get(topic, []) for change in changes: result = transform(change['document']) self.processed.append(result) print(f"Spark: Processed {len(changes)} changes") def sink_connector(self, topic, collection): """Simulate sink connector.""" count = len(self.processed) print(f"Kafka: Sink connector wrote {count} docs to {collection}") return count # Demo print("=== MongoDB + Kafka Demo ===") pipeline = MongoKafkaPipeline() # Insert into MongoDB pipeline.insert_mongo('users', {'user_id': 1, 'name': 'Alice', 'age': 30}) pipeline.insert_mongo('users', {'user_id': 2, 'name': 'Bob', 'age': 25}) pipeline.insert_mongo('users', {'user_id': 3, 'name': 'Charlie', 'age': 35}) # Source connector pipeline.source_connector('users', 'mongo.users') # Process pipeline.process('mongo.users', lambda d: {**d, 'age_group': 'senior' if d['age'] > 30 else 'adult'}) # Sink connector count = pipeline.sink_connector('mongo.users', 'analytics.enriched') print(f" Written: {count}") # KAFKA SUMMARY print("\n=== MongoDB + Kafka Summary ===") print(f"{'Component':<25} {'Description':<40}") print("-" * 65) print(f"{'Source':<25} {'MongoDB → Kafka':<40}") print(f"{'Sink':<25} {'Kafka → MongoDB':<40}") print(f"{'Change streams':<25} {'Real-time':<40}") print(f"{'CDC':<25} {'Change data capture':<40}") print(f"{'Exactly-once':<25} {'Semantics':<40}")

🚀 Performance

🎯 Fast at Scale

MongoDB performance depends on indexes, sharding, schema design, and hardware. Proper modeling is key.

Performance

Indexes: Fast queries. Sharding: Scale writes. Schema: Embed vs reference. WiredTiger: Storage engine. Read preference: Distribute. Write concern: Durability.

🚀 Indexes🚀 Sharding🚀 Schema🚀 WiredTiger
# Performance print("=== Performance Factors ===") print(""" 1. INDEXES - Speed queries - Compound for multiple - Monitor usage 2. SHARDING - Scale writes - Even distribution - Good shard key 3. SCHEMA DESIGN - Embed vs reference - Denormalize - Model for queries 4. WIREDTIGER - Storage engine - Compression - Concurrency 5. READ PREFERENCE - Distribute reads - Secondary reads 6. WRITE CONCERN - Durability - Trade-off """) print("\n=== Schema Design ===") print(""" Embed: - Data used together - One-to-few - Fast reads Reference: - Data used separately - One-to-many - Normalized Example: // Embed { user: "Alice", address: { city: "NYC", zip: "10001" } } // Reference { user: "Alice", address_id: ObjectId("...") } """) print("\n=== Explain ===") print(""" db.users.find({ city: "NYC" }).explain("executionStats") Output: - winningPlan - executionStats - index usage - docs examined - time """) print("\n=== Performance Tuning ===") print(""" 1. Index query fields 2. Use compound indexes 3. Shard for scale 4. Model for queries 5. Use read preference 6. Tune write concern 7. Monitor slow queries 8. Profile 9. Optimize storage 10. Scale hardware """) print("\n=== Monitoring ===") print(""" Tools: - mongostat - mongotop - Compass - Cloud Manager - Ops Manager Metrics: - Operations/sec - Query time - Index usage - Cache hit ratio - Connections """) # IMPLEMENTATION print("\n=== Simulating Performance ===") class MongoPerformance: """Simulate MongoDB performance.""" def __init__(self, collection_size, has_index=False, sharded=False): self.collection_size = collection_size self.has_index = has_index self.sharded = sharded def query_time(self): """Estimate query time.""" if self.has_index: # Index lookup: O(log n) import math return math.log2(self.collection_size) * 0.001 # ms else: # Full scan: O(n) return self.collection_size * 0.001 # ms def write_throughput(self): """Estimate write throughput.""" base = 10000 # ops/sec if self.sharded: base *= 10 # 10 shards return base def get_info(self): return { 'collection_size': self.collection_size, 'has_index': self.has_index, 'sharded': self.sharded, 'query_time_ms': self.query_time(), 'write_ops_sec': self.write_throughput(), } # Demo print("=== Performance Simulation ===") scenarios = [ (1000, False, False), (1000, True, False), (1000000, False, False), (1000000, True, False), (1000000, True, True), ] print(f"{'Size':<12} {'Index':<10} {'Sharded':<10} {'Query (ms)':<15} {'Write (ops/s)':<15}") print("-" * 65) for size, index, sharded in scenarios: perf = MongoPerformance(size, index, sharded) info = perf.get_info() print(f"{info['collection_size']:<12} {str(info['has_index']):<10} " f"{str(info['sharded']):<10} {info['query_time_ms']:<15.2f} " f"{info['write_ops_sec']:<15}") # PERFORMANCE SUMMARY print("\n=== Performance Summary ===") print(f"{'Factor':<25} {'Impact':<40}") print("-" * 65) print(f"{'Indexes':<25} {'High':<40}") print(f"{'Sharding':<25} {'High':<40}") print(f"{'Schema design':<25} {'High':<40}") print(f"{'WiredTiger':<25} {'Medium':<40}") print(f"{'Read preference':<25} {'Medium':<40}")

⚖️ MongoDB vs Others

🎯 Comparing Databases

MongoDB vs Cassandra, HBase, PostgreSQL, and DynamoDB for Big Data workloads.

Comparison

MongoDB: Document, flexible. Cassandra: Wide-column, scale. HBase: Hadoop. PostgreSQL: Relational. DynamoDB: Managed. Choose: Based on needs.

⚖️ MongoDB⚖️ Cassandra⚖️ HBase⚖️ PostgreSQL
# MongoDB vs Others print("=== Comparison ===") print(f"{'Aspect':<25} {'MongoDB':<20} {'Cassandra':<20} {'HBase':<20}") print("-" * 85) print(f"{'Type':<25} {'Document':<20} {'Wide-column':<20} {'Wide-column':<20}") print(f"{'Schema':<25} {'Flexible':<20} {'Flexible':<20} {'Flexible':<20}") print(f"{'Consistency':<25} {'Tunable':<20} {'Tunable':<20} {'Strong':<20}") print(f"{'Scaling':<25} {'Sharding':<20} {'Peer-to-peer':<20} {'HDFS':<20}") print(f"{'Query':<25} {'Rich':<20} {'CQL':<20} {'Limited':<20}") print(f"{'Aggregation':<25} {'Pipeline':<20} {'Limited':<20} {'No':<20}") print(f"{'Indexes':<25} {'Many':<20} {'Limited':<20} {'Row key':<20}") print(f"{'Use case':<25} {'Operational':<20} {'Time-series':<20} {'Hadoop':<20}") print("\n=== MongoDB vs PostgreSQL ===") print(f"{'Aspect':<25} {'MongoDB':<20} {'PostgreSQL':<20}") print("-" * 65) print(f"{'Type':<25} {'Document':<20} {'Relational':<20}") print(f"{'Schema':<25} {'Flexible':<20} {'Fixed':<20}") print(f"{'ACID':<25} {'Multi-doc':<20} {'Full':<20}") print(f"{'Joins':<25} {'$lookup':<20} {'Native':<20}") print(f"{'Scaling':<25} {'Sharding':<20} {'Vertical':<20}") print(f"{'Use case':<25} {'Operational':<20} {'OLTP + Analytics':<20}") print("\n=== MongoDB vs DynamoDB ===") print(f"{'Aspect':<25} {'MongoDB':<20} {'DynamoDB':<20}") print("-" * 65) print(f"{'Type':<25} {'Document':<20} {'Key-value':<20}") print(f"{'Managed':<25} {'Self/Atlas':<20} {'Fully managed':<20}") print(f"{'Scaling':<25} {'Manual/Auto':<20} {'Auto':<20}") print(f"{'Query':<25} {'Rich':<20} {'Limited':<20}") print(f"{'Cost':<25} {'Variable':<20} {'Pay-per-use':<20}") print(f"{'Lock-in':<25} {'Low':<20} {'High':<20}") print("\n=== When to Use Which ===") print(""" USE MONGODB: ✓ Flexible schema ✓ Rich queries ✓ Aggregation ✓ Operational data ✓ Real-time USE CASSANDRA: ✓ Time-series ✓ High write ✓ Multi-region ✓ Tunable consistency USE HBASE: ✓ Hadoop ecosystem ✓ HDFS storage ✓ Random access USE POSTGRESQL: ✓ OLTP ✓ Complex joins ✓ ACID ✓ Analytics USE DYNAMODB: ✓ AWS-native ✓ Serverless ✓ Managed ✓ Key-value """)

💼 Use Cases

🎯 Real-World Applications

MongoDB powers operational applications, real-time analytics, IoT, and more across industries.

Use Cases

Catalog: Product data. IoT: Sensor data. Personalization: User profiles. Logs: Application logs. Real-time: Analytics. CMS: Content.

💼 Catalog💼 IoT💼 Personalization💼 Logs
# Use Cases print("=== Industry Use Cases ===") print(f"{'Industry':<20} {'Use Case':<35} {'Scale':<20}") print("-" * 75) print(f"{'E-commerce':<20} {'Product catalog':<35} {'Millions':<20}") print(f"{'IoT':<20} {'Sensor data':<35} {'Billions':<20}") print(f"{'Media':<20} {'User profiles':<35} {'Millions':<20}") print(f"{'Finance':<20} {'Transaction logs':<35} {'Billions':<20}") print(f"{'Tech':<20} {'Application data':<35} {'Millions':<20}") print(f"{'Healthcare':<20} {'Patient records':<35} {'Millions':<20}") print("\n=== 1. Product Catalog ===") print(""" Use case: - Product data - Varied attributes - Flexible schema Why MongoDB: - Different products have different fields - Fast reads - Rich queries Example: { name: "Laptop", brand: "Apple", specs: { ram: "16GB", storage: "512GB" }, price: 1999, tags: ["electronics", "computers"] } """) print("\n=== 2. IoT ===") print(""" Use case: - Sensor data - Time-series - Real-time Why MongoDB: - High write throughput - Flexible schema - Sharding - TTL indexes Example: { device_id: "sensor_1", timestamp: ISODate(), temperature: 25.5, humidity: 60 } """) print("\n=== 3. Personalization ===") print(""" Use case: - User profiles - Preferences - Behavior Why MongoDB: - Flexible schema - Fast reads - Rich queries Example: { user_id: 123, preferences: { theme: "dark", language: "en" }, behavior: { last_login: ISODate(), pages: [...] } } """) print("\n=== 4. Logs ===") print(""" Use case: - Application logs - Real-time - Search Why MongoDB: - High write - Flexible schema - Text search - TTL Example: { timestamp: ISODate(), level: "ERROR", message: "...", service: "api" } """) print("\n=== 5. Real-Time Analytics ===") print(""" Use case: - Aggregations - Dashboards - Alerts Why MongoDB: - Aggregation pipeline - Sharding - Spark integration Example: db.events.aggregate([ { $match: { date: { $gte: today } } }, { $group: { _id: "$type", count: { $sum: 1 } } } ]) """) # IMPLEMENTATION print("\n=== Practical Example: IoT Pipeline ===") class IoTPipeline: """Simulate IoT pipeline.""" def __init__(self): self.devices = {} self.readings = [] def register_device(self, device_id): """Register device.""" self.devices[device_id] = {'id': device_id, 'status': 'active'} print(f"IoT: Registered {device_id}") def record_reading(self, device_id, temperature, humidity): """Record reading.""" reading = { 'device_id': device_id, 'timestamp': time.time(), 'temperature': temperature, 'humidity': humidity, } self.readings.append(reading) return reading def aggregate(self): """Aggregate readings.""" if not self.readings: return {} temps = [r['temperature'] for r in self.readings] return { 'count': len(self.readings), 'avg_temp': sum(temps) / len(temps), 'max_temp': max(temps), 'min_temp': min(temps), } # Demo print("=== IoT Pipeline Demo ===") iot = IoTPipeline() # Register devices iot.register_device('sensor_1') iot.register_device('sensor_2') # Record readings import random for i in range(10): reading = iot.record_reading( f'sensor_{i % 2 + 1}', temperature=20 + random.random() * 10, humidity=50 + random.random() * 20, ) # Aggregate print(f"\nAggregated: {iot.aggregate()}") # USE CASE SUMMARY print("\n=== Use Case Summary ===") print(f"{'Use Case':<25} {'Description':<40}") print("-" * 65) print(f"{'Catalog':<25} {'Product data':<40}") print(f"{'IoT':<25} {'Sensor data':<40}") print(f"{'Personalization':<25} {'User profiles':<40}") print(f"{'Logs':<25} {'Application logs':<40}") print(f"{'Real-time':<25} {'Analytics':<40}")

✅ Best Practices

🎯 Doing It Right

Follow best practices for schema design, indexing, sharding, and operations to get the most out of MongoDB.

Best Practices

Schema: Model for queries. Indexes: Cover queries. Sharding: Good key. Monitoring: Watch metrics. Backup: Regular. Security: Enable auth.

✅ Schema✅ Indexes✅ Sharding✅ Monitoring
# Best Practices print("=== Schema Design ===") print(""" 1. MODEL FOR QUERIES - What queries? - What data together? - Embed vs reference 2. EMBED WHEN - Data used together - One-to-few - Fast reads 3. REFERENCE WHEN - Data used separately - One-to-many - Normalized 4. AVOID - Unbounded arrays - Deep nesting - Large documents """) print("\n=== Indexing ===") print(""" 1. INDEX QUERY FIELDS 2. USE COMPOUND INDEXES 3. PREFIX MATTERS 4. AVOID OVER-INDEXING 5. MONITOR USAGE 6. USE EXPLAIN 7. CONSIDER CARDINALITY 8. COVER QUERIES """) print("\n=== Sharding ===") print(""" 1. GOOD SHARD KEY - High cardinality - Even distribution - Common query field 2. AVOID - Monotonic keys - Low cardinality - Rarely queried 3. CONSIDER - Hashed for even - Ranged for queries - Compound for balance """) print("\n=== Operations ===") print(""" 1. ENABLE AUTHENTICATION 2. USE TLS 3. REGULAR BACKUPS 4. MONITOR METRICS 5. SET UP ALERTS 6. PLAN CAPACITY 7. UPGRADE REGULARLY 8. DOCUMENT """) print("\n=== Security ===") print(""" 1. Enable auth security.authorization: enabled 2. Create users db.createUser({ user: "admin", pwd: "password", roles: ["root"] }) 3. Use TLS net.tls.mode: requireTLS 4. Role-based access - read, readWrite - dbAdmin, userAdmin - clusterAdmin 5. Encryption at rest - WiredTiger encryption - Cloud KMS """) print("\n=== Anti-Patterns ===") print(""" AVOID: ✗ Unbounded arrays ✗ Massive documents ✗ Deep nesting ✗ No indexes ✗ Bad shard key ✗ Ignoring monitoring ✗ No backups ✗ No auth """) # BEST PRACTICES CHECKLIST print("\n=== Checklist ===") print(""" SCHEMA: ✓ Model for queries ✓ Embed vs reference ✓ Avoid unbounded INDEXES: ✓ Query fields ✓ Compound ✓ Monitor SHARDING: ✓ Good key ✓ Even distribution ✓ Monitor OPS: ✓ Backup ✓ Monitor ✓ Alert ✓ Upgrade SECURITY: ✓ Auth ✓ TLS ✓ Encryption ✓ Roles """)

🚀 Future

🎯 Where MongoDB Is Going

MongoDB is evolving: cloud-native (Atlas), AI integration, better analytics, and unified data platform.

Future Directions

Atlas: Managed cloud. AI: Vector search. Analytics: Columnar. Unified: One platform. Serverless: Atlas. Edge: Mobile.

🚀 Atlas🚀 AI🚀 Analytics🚀 Unified
# Future of MongoDB print("=== 1. MongoDB Atlas ===") print(""" Atlas: - Fully managed - Multi-cloud - Auto-scaling - Global clusters Features: - Atlas Search - Atlas Data Lake - Atlas Charts - Atlas SQL Future: - Serverless - Multi-cloud - Global """) print("\n=== 2. AI and Vector Search ===") print(""" Vector Search: - Embeddings - Similarity search - RAG - AI applications Features: - Atlas Vector Search - Native integration - LangChain - LLM support Future: - AI-native - Vector indexes - RAG pipelines """) print("\n=== 3. Analytics ===") print(""" Analytics: - Columnar storage - Atlas SQL - BI connectors - Data Lake Features: - Atlas Data Lake - Atlas SQL - BI connectors - Spark connector Future: - Unified analytics - Real-time - Lakehouse """) print("\n=== 4. Serverless ===") print(""" Serverless: - Atlas Serverless - Pay per use - Auto-scaling - No infrastructure Future: - Serverless default - Event-driven - Cost-effective """) print("\n=== 5. Edge and Mobile ===") print(""" Edge: - Realm - Mobile sync - Offline-first - Edge computing Features: - Realm Sync - Atlas Device Sync - Mobile SDKs Future: - Edge-native - Offline - Sync """) print("\n=== 6. Unified Platform ===") print(""" Unified: - Operational - Analytics - Search - Vector - Mobile - Edge Future: - One platform - All workloads - Unified data """) # TIMELINE print("\n=== Timeline ===") print(""" Past: 2009: MongoDB created 2010: First release 2013: Sharding 2016: Atlas 2020: Atlas Search 2023: Vector Search Present: 2024: Unified platform 2025: AI-native Future: 2030: AI-native default? 2035: Edge? 2040: New paradigms? """) print("\n=== Final Thoughts ===") print(""" MongoDB is: - Document database - Distributed - Scalable - Flexible - Evolving The future is: - Atlas - AI - Analytics - Unified - Serverless "MongoDB: the database for modern applications." Keep exploring. Keep building. """)