🧠 ZooKeeper Fundamentals

Q: What is Apache ZooKeeper?

Apache ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. It's used by Kafka, HBase, Hadoop, and many distributed systems for coordination.

Q: What is a znode?

A znode is a node in ZooKeeper's hierarchical namespace, similar to a file or directory in a filesystem. Znodes store data, have a path, and can be persistent, ephemeral, sequential, or a combination.

Q: What is a watch in ZooKeeper?

A watch is a one-time trigger that clients set on znodes to receive notifications when the znode changes (created, deleted, data changed, children changed). Watches enable clients to react to changes without polling.

Q: How does ZooKeeper achieve consensus?

ZooKeeper uses the ZAB (ZooKeeper Atomic Broadcast) protocol, a variant of Paxos, to achieve consensus among ensemble members. A leader is elected, and all writes go through the leader, which broadcasts to followers.

Q: Does Kafka still need ZooKeeper?

No. Since Kafka 2.8, KRaft mode allows Kafka to run without ZooKeeper. Kafka 3.x supports both; Kafka 4.0 (2025) removed ZooKeeper entirely. However, many existing deployments still use ZooKeeper.

🎯 What is Apache ZooKeeper?

Apache ZooKeeper is a centralized coordination service for distributed systems. It provides a reliable, ordered, and highly available hierarchical key-value store, enabling distributed systems to coordinate through primitives like leader election, configuration management, naming, distributed locks, and group membership. It's used by Kafka, HBase, Hadoop, and many others.

Why ZooKeeper Matters

Coordination: Distributed consensus. Reliable: Highly available. Ordered: Total order. Fast: In-memory. Simple: Hierarchical namespace. Foundation: Many systems use it.

📌 Coordination📌 Consensus📌 Leader Election📌 Configuration

# Apache ZooKeeper — the big picture
# Distributed coordination service.

# CONCEPTUAL OVERVIEW
print("""
=== Apache ZooKeeper ===

What it is:
  - Coordination service
  - Hierarchical namespace
  - Reliable, ordered, fast
  - Distributed consensus
  - Watches for changes

Used by:
  - Kafka (leader election)
  - HBase (region assignment)
  - Hadoop (HA)
  - Solr (cluster state)
  - Dubbo (service discovery)

Key features:
  - Znodes (nodes)
  - Watches (notifications)
  - Sessions
  - Sequential consistency
  - Atomicity
  - Reliability
  - Timeliness
""")

# WHY IT MATTERS
print("=== Why It Matters ===")
print("""
Distributed challenges:
  - Coordination
  - Leader election
  - Configuration
  - Naming
  - Locking
  - Membership

ZooKeeper solves:
  - Simple API
  - Strong consistency
  - High availability
  - Ordered updates
  - Watches for changes
""")

# REAL-WORLD
print("=== Real-World Scale ===")
print("""
Companies using ZooKeeper:
  - LinkedIn
  - Yahoo (created it)
  - Netflix
  - Facebook
  - Twitter
  - Adobe
  - Uber
  - Airbnb

Scale:
  - Thousands of nodes
  - Millions of ops/sec
  - Years of uptime
""")

🏗️ Architecture

🎯 ZooKeeper Architecture

ZooKeeper uses a leader-follower architecture with an ensemble of servers. The leader handles writes, followers serve reads, and consensus is achieved via the ZAB protocol.

Architecture

Ensemble: Group of servers. Leader: Writes. Follower: Reads. Observer: Read-only. Quorum: Majority. ZAB: Consensus protocol.

🏗️ Ensemble🏗️ Leader🏗️ Follower🏗️ Quorum

# ZooKeeper Architecture
print("=== Architecture Diagram ===")
print("""
  ┌─────────────────────────────────────────┐
  │              Clients                    │
  │         (Apps, Kafka, HBase)            │
  └─────────────────┬───────────────────────┘
                    │
        ┌───────────┼───────────┐
        │           │           │
        ▼           ▼           ▼
  ┌─────────┐ ┌─────────┐ ┌─────────┐
  │  ZK 1   │ │  ZK 2   │ │  ZK 3   │
  │(Leader) │ │(Follower│ │(Follower│
  │         │ │  )      │ │  )      │
  │  Writes │ │  Reads  │ │  Reads  │
  └────┬────┘ └────┬────┘ └────┬────┘
       │           │           │
       └───────────┼───────────┘
                   │
              ZAB Protocol
              (Consensus)
                   │
                   ▼
  ┌─────────────────────────────────────────┐
  │           Data Tree                     │
  │         (In-memory)                     │
  │         (Persistent via log)            │
  └─────────────────────────────────────────┘
""")

print("\n=== Components ===")
print("""
1. LEADER
   - One per ensemble
   - Handles all writes
   - Broadcasts to followers
   - Elected via ZAB

2. FOLLOWER
   - Multiple per ensemble
   - Serve reads
   - Forward writes to leader
   - Vote in elections

3. OBSERVER
   - Like follower, no vote
   - Read-only
   - Scales reads
   - Doesn't affect quorum

4. ENSEMBLE
   - Group of servers
   - Usually 3, 5, or 7
   - Odd number (quorum)
   - Tolerates (n-1)/2 failures

5. QUORUM
   - Majority of ensemble
   - (n/2) + 1
   - Required for writes
   - 3 nodes → 2 quorum
   - 5 nodes → 3 quorum

6. DATA TREE
   - In-memory
   - Backed by log
   - Snapshots
   - Persistent
""")

print("\n=== Reads vs Writes ===")
print("""
Reads:
  - Served by any server
  - Fast (in-memory)
  - May be slightly stale
  - Scale with followers/observers

Writes:
  - Go through leader
  - Consensus required
  - Ordering guaranteed
  - Slower than reads

Linearizable writes:
  - All writes ordered
  - Consistent across ensemble
""")

print("\n=== Ensemble Sizes ===")
print(f"{'Size':<10} {'Quorum':<10} {'Tolerates':<15} {'Use Case':<25}")
print("-" * 65)
print(f"{'1':<10} {'1':<10} {'0':<15} {'Dev only':<25}")
print(f"{'3':<10} {'2':<10} {'1':<15} {'Production':<25}")
print(f"{'5':<10} {'3':<10} {'2':<15} {'Production':<25}")
print(f"{'7':<10} {'4':<10} {'3':<15} {'Large scale':<25}")

print("\n=== Configuration ===")
print("""
# zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/var/lib/zookeeper
clientPort=2181
maxClientCnxns=60

# Ensemble
server.1=zk1:2888:3888
server.2=zk2:2888:3888
server.3=zk3:2888:3888
""")

📁 Znodes

🎯 The Data Model

A znode is a node in ZooKeeper's hierarchical namespace, like a file or directory. Znodes store data (up to 1 MB), have a path, and support different lifetimes and ordering.

Znodes

Persistent: Until deleted. Ephemeral: Session-bound. Sequential: Auto-numbered. Data: Up to 1MB. Version: Optimistic locking. Watch: Notify changes.

📁 Persistent📁 Ephemeral📁 Sequential📁 Version

# Znodes
print("=== What is a Znode? ===")
print("""
Znode:
  - Node in tree
  - Like file/directory
  - Stores data
  - Has path
  - Has version
  - Can have children

Structure:
  /
  ├── zookeeper
  │   └── config
  ├── kafka
  │   ├── brokers
  │   │   ├── ids
  │   │   └── topics
  │   └── controller
  └── hbase
      ├── meta-region-server
      └── rs
""")

print("\n=== Znode Types ===")
print("""
1. PERSISTENT
   - Survives session end
   - Until explicitly deleted
   - Default type
   
   create /myapp "data"

2. EPHEMERAL
   - Deleted on session end
   - Tied to session
   - For locks, membership
   
   create -e /myapp/lock "owner"

3. PERSISTENT_SEQUENTIAL
   - Persistent + auto-numbered
   - Monotonic counter
   - For queues, ordering
   
   create -s /myapp/task "task1"
   → /myapp/task0000000001

4. EPHEMERAL_SEQUENTIAL
   - Ephemeral + auto-numbered
   - For locks with queue
   - For leader election
   
   create -e -s /myapp/lock "owner"
   → /myapp/lock0000000001
""")

print("\n=== Znode Attributes ===")
print("""
Stat:
  - czxid: Create zxid
  - mzxid: Last modified zxid
  - ctime: Create time
  - mtime: Modify time
  - version: Data version
  - cversion: Children version
  - aversion: ACL version
  - ephemeralOwner: Session ID (if ephemeral)
  - dataLength: Data size
  - numChildren: Children count
  - pzxid: Last child zxid
""")

print("\n=== Data Limits ===")
print("""
Data size:
  - Max 1 MB per znode
  - Recommended <1 KB
  - Metadata only
  - Not for large data

Why:
  - In-memory
  - Replicated
  - Fast
  - Not designed for bulk data

Use for:
  - Configuration
  - Metadata
  - State
  - Coordination
""")

print("\n=== Versioning ===")
print("""
Version:
  - Each znode has version
  - Incremented on update
  - Optimistic locking
  - CAS-style operations

Example:
  # Read
  data, stat = zk.get("/myapp")
  version = stat.version
  
  # Update with version check
  zk.set("/myapp", "new_data", version=version)
  
  # Fails if version changed
""")

print("\n=== ACL ===")
print("""
Access Control:
  - Per znode
  - ACL list
  - Permissions: READ, WRITE, CREATE, DELETE, ADMIN
  - Schemes: world, auth, digest, ip, sasl

Example:
  from kazoo.security import make_acl
  
  acl = [make_acl("auth", "", all=True)]
  zk.create("/myapp", "data", acl=acl)
""")

print("\n=== Example: Kafka ===")
print("""
Kafka uses ZooKeeper for:

/brokers
  /ids
    /0  → broker metadata
    /1  → broker metadata
  /topics
    /my-topic
      /partitions
        /0  → partition state
        /1  → partition state

/controller → leader broker
/admin
  /delete_topics
/config
  /topics
    /my-topic → config
""")

👀 Watches

🎯 Change Notifications

A watch is a one-time trigger set on a znode. When the znode changes, ZooKeeper notifies the client. Watches enable reactive behavior without polling.

Watches

One-time: Fire once. Async: Notifications. Ordered: Total order. Light: No polling. Reliable: Guaranteed. Re-register: After trigger.

👀 One-Time👀 Async👀 Ordered👀 Re-register

# Watches
print("=== What is a Watch? ===")
print("""
Watch:
  - Trigger on znode change
  - One-time
  - Async notification
  - Client must re-register
  - Total order

Events:
  - Created
  - Deleted
  - Data changed
  - Children changed

Types:
  - Data watch (getData, exists)
  - Child watch (getChildren)
""")

print("\n=== Why One-Time? ===")
print("""
One-time:
  - Prevents memory leak
  - Client controls
  - No overload
  - Explicit

Client must:
  - Re-register after trigger
  - Or use recursive watches
  - Or use curator (handles)
""")

print("\n=== Example ===")
print("""
from kazoo.client import KazooClient

zk = KazooClient(hosts='localhost:2181')
zk.start()

# Data watch
def watch_data(event):
    print(f"Event: {event}")
    # Re-register
    zk.get("/myapp", watch=watch_data)

zk.get("/myapp", watch=watch_data)

# Children watch
def watch_children(event):
    print(f"Children event: {event}")
    zk.get_children("/myapp", watch=watch_children)

zk.get_children("/myapp", watch=watch_children)

# Wait
import time
time.sleep(60)

zk.stop()
""")

print("\n=== Guarantees ===")
print("""
Watches are:
  - Ordered (client sees in order)
  - Reliable (guaranteed before client learns)
  - One-time
  - Fast (async)
  - Data consistent

Order:
  1. Client reads znode
  2. Sets watch
  3. Change occurs
  4. Client notified
  5. Before next read

Guarantee:
  - Client sees change event
  - Before seeing new data
""")

print("\n=== Curator ===")
print("""
Apache Curator:
  - High-level ZooKeeper client
  - Manages watches automatically
  - Re-registers on trigger
  - Recipes built-in

Example:
  from curator import CuratorFrameworkFactory
  
  client = CuratorFrameworkFactory.newClient(
      "localhost:2181",
      RetryNTimes(3, 1000)
  )
  client.start()
  
  # Watch
  client.getData().watched().forPath("/myapp")
""")

print("\n=== Use Cases ===")
print("""
1. CONFIGURATION
   - Watch config znode
   - Update on change
   - No restart

2. SERVICE DISCOVERY
   - Watch /services
   - React to new services
   - React to failures

3. LEADER ELECTION
   - Watch leader znode
   - React to leader change

4. CLUSTER MEMBERSHIP
   - Watch /members
   - React to join/leave

5. DISTRIBUTED LOCK
   - Watch lock znode
   - Acquire when released

6. QUEUE
   - Watch queue znode
   - Process when items added
""")

🔑 Sessions

🎯 Client-Server Connection

A session is a connection between a client and the ZooKeeper ensemble. Sessions have a timeout and are used to manage ephemeral znodes and watch delivery.

Sessions

Connection: Client-server. Timeout: Configurable. Heartbeat: Keep-alive. Ephemeral: Session-bound. Reconnect: Same session. Expire: Clean up.

🔑 Connection🔑 Timeout🔑 Ephemeral🔑 Reconnect

# Sessions
print("=== What is a Session? ===")
print("""
Session:
  - Client connects to ensemble
  - Gets session ID
  - Has timeout
  - Heartbeats
  - State

Properties:
  - sessionId: 64-bit ID
  - timeout: milliseconds
  - password: for reconnection
  - state: CONNECTING, CONNECTED, etc.
""")

print("\n=== Session Timeout ===")
print("""
Timeout:
  - Client configures
  - Server enforces bounds
  - minSessionTimeout
  - maxSessionTimeout

Default:
  - Client: 10 seconds
  - Server min: 2 × tickTime
  - Server max: 20 × tickTime
  - tickTime: 2000 ms

Effect:
  - Too short: false expiration
  - Too long: slow failure detection

Recommendation:
  - 10-30 seconds
  - Balance sensitivity
""")

print("\n=== Session States ===")
print("""
States:
  - CONNECTING
  - CONNECTED
  - CONNECTEDREADONLY
  - CLOSED
  - AUTH_FAILED
  - NOT_CONNECTED

Transitions:
  CONNECTING → CONNECTED
  CONNECTED → NOT_CONNECTED
  NOT_CONNECTED → CONNECTED
  NOT_CONNECTED → CLOSED
""")

print("\n=== Ephemeral Znodes ===")
print("""
Ephemeral:
  - Tied to session
  - Deleted on session end
  - Auto cleanup
  - Perfect for locks

Example:
  # Create ephemeral znode
  zk.create("/lock", ephemeral=True)
  
  # If client crashes:
  #   - Session expires
  #   - Znode deleted
  #   - Lock released
  
  # Automatic!
""")

print("\n=== Reconnection ===")
print("""
Reconnect:
  - Client reconnects
  - Same session if not expired
  - Ephemeral znodes intact
  - Watches intact

If expired:
  - New session
  - Ephemeral znodes gone
  - Watches lost

Handling:
  - Listen for state changes
  - Re-register watches
  - Recreate ephemeral
""")

print("\n=== Example ===")
print("""
from kazoo.client import KazooClient
from kazoo.client import KazooState

zk = KazooClient(hosts='localhost:2181')
zk.start()

def listener(state):
    if state == KazooState.LOST:
        print("Session lost")
        # Re-establish
    elif state == KazooState.SUSPENDED:
        print("Disconnected")
        # Buffer
    elif state == KazooState.CONNECTED:
        print("Connected")
        # Resume

zk.add_listener(listener)

# Session info
session_id = zk.client_id[0]
print(f"Session ID: {session_id}")

# Create ephemeral
zk.create("/myapp/lock", ephemeral=True)
""")

print("\n=== Best Practices ===")
print("""
1. Set reasonable timeout
   - 10-30 seconds

2. Handle state changes
   - Listen for LOST, SUSPENDED, CONNECTED
   - Re-register
   - Recreate ephemeral

3. Use ephemeral for locks
   - Auto cleanup

4. Monitor session count
   - Avoid overload

5. Close sessions
   - Graceful shutdown
""")

🤝 Consensus (ZAB)

🎯 ZooKeeper Atomic Broadcast

ZAB (ZooKeeper Atomic Broadcast) is ZooKeeper's consensus protocol. It ensures all servers agree on the order of updates and provides total order for writes.

ZAB Protocol

Consensus: Agreement. Total order: All servers. Leader: Proposes. Followers: Acknowledge. Quorum: Majority. Recovery: Leader election.

🤝 Consensus🤝 Total Order🤝 Quorum🤝 Recovery

# ZAB Protocol
print("=== What is ZAB? ===")
print("""
ZAB (ZooKeeper Atomic Broadcast):
  - Consensus protocol
  - Variant of Paxos
  - Total order
  - Atomic broadcast
  - Primary-backup

Ensures:
  - Agreement
  - Total order
  - Reliability
  - Leader election
  - Recovery
""")

print("\n=== ZAB Phases ===")
print("""
1. LEADER ELECTION
   - Detect leader failure
   - Elect new leader
   - Quorum required

2. DISCOVERY
   - New leader collects state
   - Synchronize with followers
   - Ensure consistency

3. SYNCHRONIZATION
   - Followers sync
   - Same state as leader
   - Ready to serve

4. BROADCAST
   - Normal operation
   - Leader proposes
   - Followers acknowledge
   - Commit on quorum
""")

print("\n=== Broadcast Protocol ===")
print("""
Write flow:
  1. Client → Leader
  2. Leader assigns zxid
  3. Leader → Followers (PROPOSE)
  4. Followers → Leader (ACK)
  5. Quorum reached
  6. Leader → Followers (COMMIT)
  7. Followers apply
  8. Leader → Client (response)

zxid:
  - 64-bit ID
  - Epoch (32) + Counter (32)
  - Total order
  - Unique
""")

print("\n=== Quorum ===")
print("""
Quorum:
  - Majority of ensemble
  - (n/2) + 1
  - Required for write
  - Required for election

Examples:
  3 nodes → 2
  5 nodes → 3
  7 nodes → 4

Guarantees:
  - Writes survive
  - Any quorum overlap
  - Consistent
""")

print("\n=== Leader Election ===")
print("""
Election:
  - Leader fails
  - Followers detect
  - Election
  - New leader

Protocols:
  - Fast Leader Election (FLE)
  - Default
  - Based on zxid + server ID

Priority:
  1. Higher zxid (more recent)
  2. Higher server ID
  3. Majority
""")

print("\n=== Guarantees ===")
print("""
ZAB guarantees:
  1. Total order
     - All servers see same order

  2. Atomicity
     - All or nothing

  3. Reliability
     - No lost updates

  4. Timeliness
     - Bounded delay

  5. Causal order
     - Related updates ordered

These are:
  - Linearizable writes
  - Sequential consistency
  - High availability
""")

print("\n=== Comparison with Paxos ===")
print("""
ZAB vs Paxos:
  - ZAB: Primary-backup
  - Paxos: Symmetric

ZAB:
  - Leader elected
  - Leader broadcasts
  - Simpler
  - Higher throughput

Paxos:
  - Any proposer
  - More flexible
  - More complex

Both:
  - Consensus
  - Quorum
  - Safety
  - Liveness
""")

👑 Leader Election

🎯 Choosing a Leader

Leader election is a key ZooKeeper recipe. In a distributed system, you often need one node to coordinate. ZooKeeper's sequential ephemeral znodes make this trivial.

Leader Election

Ephemeral: Session-bound. Sequential: Auto-numbered. Smallest: Leader. Watch: Next in line. Auto: Failover. Simple: Recipe.

👑 Ephemeral👑 Sequential👑 Smallest👑 Failover

# Leader Election
print("=== What is Leader Election? ===")
print("""
Leader Election:
  - Choose one node as leader
  - Others as followers
  - Automatic failover
  - Coordination primitive

Use cases:
  - Kafka controller
  - HBase master
  - Distributed databases
  - Cluster coordination
  - Job scheduling
""")

print("\n=== How It Works ===")
print("""
1. Each candidate creates:
   /election/n_0000000001 (ephemeral, sequential)

2. Get children:
   /election/n_0000000001
   /election/n_0000000002
   /election/n_0000000003

3. Smallest = leader
   Others = followers

4. Each follower watches
   the node before it

5. If leader fails:
   Its node deleted
   Next in line becomes leader
""")

print("\n=== Example (Kazoo) ===")
print("""
from kazoo.client import KazooClient
from kazoo.recipe.election import Election

zk = KazooClient(hosts='localhost:2181')
zk.start()

def leader_func():
    print("I am the leader!")
    # Do leader work

election = Election(zk, "/election", "candidate1")
election.run(leader_func)
""")

print("\n=== Example (Curator) ===")
print("""
from curator import CuratorFrameworkFactory
from curator.recipes import LeaderLatch

client = CuratorFrameworkFactory.newClient("localhost:2181")
client.start()

latch = LeaderLatch(client, "/election", "candidate1")
latch.start()

if latch.hasLeadership():
    print("I am the leader!")

# Release
latch.close()
""")

print("\n=== Manual Implementation ===")
print("""
from kazoo.client import KazooClient

zk = KazooClient(hosts='localhost:2181')
zk.start()

# Create election path
zk.ensure_path("/election")

# Create ephemeral sequential
my_node = zk.create(
    "/election/n_",
    ephemeral=True,
    sequence=True
)

def check_leader():
    children = sorted(zk.get_children("/election"))
    my_name = my_node.split("/")[-1]
    
    if children[0] == my_name:
        return True
    else:
        # Watch the node before me
        idx = children.index(my_name)
        prev = children[idx - 1]
        zk.get(f"/election/{prev}", watch=watch_prev)
        return False

def watch_prev(event):
    if check_leader():
        become_leader()

def become_leader():
    print("I am the leader!")
""")

print("\n=== Guarantees ===")
print("""
1. Only one leader
   - Smallest znode

2. Automatic failover
   - Ephemeral deleted
   - Next becomes leader

3. No split brain
   - Sequential ordering
   - Total order

4. Fast detection
   - Session timeout
   - Watch notification

5. Scalable
   - Many followers
   - Efficient
""")

print("\n=== Best Practices ===")
print("""
1. Use libraries
   - Curator
   - Kazoo
   - Don't reinvent

2. Handle session
   - Listen for LOST
   - Re-enter election

3. Leader work
   - Small
   - Fast
   - Resilient

4. Follower work
   - Watch
   - Ready to take over

5. Test failover
   - Kill leader
   - Verify

6. Monitor
   - Leader changes
   - Election duration
""")

📖 Coordination Recipes

🎯 Common Patterns

ZooKeeper recipes are common coordination patterns built on ZooKeeper primitives. They solve distributed systems problems elegantly.

Recipes

Leader: Election. Lock: Mutual exclusion. Queue: FIFO. Barrier: Sync. Membership: Group. Config: Dynamic.

📖 Leader📖 Lock📖 Queue📖 Barrier

# Coordination Recipes
print("=== 1. Leader Election ===")
print("""
Leader Election:
  - Ephemeral sequential znodes
  - Smallest = leader
  - Watch predecessor
  - Auto failover

See previous section.
""")

print("\n=== 2. Distributed Lock ===")
print("""
Distributed Lock:
  - Similar to leader election
  - But releases lock
  - Acquire/release

Implementation:
  - Create ephemeral sequential
  - Smallest = lock holder
  - Watch predecessor
  - Delete on release

Example (Kazoo):
  from kazoo.recipe.lock import Lock
  
  lock = Lock(zk, "/lock")
  with lock:
      # Critical section
      pass
""")

print("\n=== 3. Distributed Queue ===")
print("""
Distributed Queue:
  - FIFO
  - Producers add
  - Consumers remove
  - Ordered

Implementation:
  - Create persistent sequential
  - Add task: create /queue/task_n
  - Consume: get children, take smallest
  - Delete after processing

Example (Kazoo):
  from kazoo.recipe.queue import Queue
  
  queue = Queue(zk, "/queue")
  queue.put("task")
  task = queue.get()
""")

print("\n=== 4. Barrier ===")
print("""
Barrier:
  - Wait for N processes
  - Then proceed
  - Sync point

Implementation:
  - Create ephemeral sequential under /barrier
  - Get children count
  - If count >= N: proceed
  - Else: watch

Example (Kazoo):
  from kazoo.recipe.barrier import Barrier
  
  barrier = Barrier(zk, "/barrier", 5)
  barrier.wait()
""")

print("\n=== 5. Group Membership ===")
print("""
Group Membership:
  - Join/leave group
  - Watch members
  - React to changes

Implementation:
  - Create ephemeral sequential under /members
  - Watch children
  - Session end → auto leave

Example:
  # Join
  zk.create("/members/m_", ephemeral=True, sequence=True)
  
  # List
  members = zk.get_children("/members")
  
  # Watch
  zk.get_children("/members", watch=watch_members)
""")

print("\n=== 6. Configuration Management ===")
print("""
Config Management:
  - Dynamic configuration
  - Watch changes
  - No restart

Implementation:
  - Create persistent znode
  - Store config
  - Watch for changes
  - Reload

Example:
  # Read config
  data, stat = zk.get("/config/app")
  config = json.loads(data)
  
  # Watch
  zk.get("/config/app", watch=reload_config)
""")

print("\n=== 7. Two-Phase Commit ===")
print("""
Two-Phase Commit:
  - Coordinator
  - Participants
  - Commit/abort

Implementation:
  - Coordinator creates /txn/txid
  - Participants vote
  - Coordinator decides
  - Notify all

Complex:
  - Usually use higher-level
  - Curator has recipes
""")

print("\n=== 8. Service Discovery ===")
print("""
Service Discovery:
  - Register services
  - Discover
  - Health check

Implementation:
  - Create /services/{name}/instances/{id}
  - Ephemeral
  - Watch for changes

Example:
  # Register
  zk.create(f"/services/myapp/instances/{instance_id}",
            value=json.dumps({"host": host, "port": port}),
            ephemeral=True)
  
  # Discover
  instances = zk.get_children("/services/myapp/instances")
""")

print("\n=== Curator Recipes ===")
print("""
Curator (Netflix):
  - High-level API
  - Tested recipes
  - Recommended

Recipes:
  - LeaderLatch
  - LeaderSelector
  - InterProcessMutex
  - InterProcessSemaphore
  - DistributedQueue
  - DistributedBarrier
  - ServiceDiscovery
  - NodeCache
  - PathChildrenCache
  - TreeCache

Example:
  from curator.recipes import InterProcessMutex
  
  lock = InterProcessMutex(client, "/lock")
  if lock.acquire(timeout=5):
      try:
          # Critical section
          pass
      finally:
          lock.release()
""")

print("\n=== Comparison ===")
print(f"{'Recipe':<25} {'Complexity':<15} {'Use Case':<30}")
print("-" * 70)
print(f"{'Leader Election':<25} {'Low':<15} {'Cluster coordination':<30}")
print(f"{'Distributed Lock':<25} {'Low':<15} {'Mutual exclusion':<30}")
print(f"{'Distributed Queue':<25} {'Low':<15} {'Task queue':<30}")
print(f"{'Barrier':<25} {'Low':<15} {'Sync point':<30}")
print(f"{'Group Membership':<25} {'Low':<15} {'Cluster state':<30}")
print(f"{'Config Management':<25} {'Low':<15} {'Dynamic config':<30}")
print(f"{'Two-Phase Commit':<25} {'High':<15} {'Transactions':<30}")
print(f"{'Service Discovery':<25} {'Low':<15} {'Microservices':<30}")

⌨️ CLI & Commands

🎯 ZooKeeper Shell

ZooKeeper's CLI (zkCli.sh) lets you interact with the ensemble, create/read/delete znodes, and inspect state.

CLI

Connect: zkCli. ls: List. get: Read. create: Write. delete: Remove. stat: Info.

⌨️ Connect⌨️ ls⌨️ get⌨️ create

# ZooKeeper CLI
print("=== Connect ===")
print("""
# Local
$ zkCli.sh

# Remote
$ zkCli.sh -server localhost:2181
$ zkCli.sh -server zk1:2181,zk2:2181,zk3:2181

# With auth
$ zkCli.sh -server localhost:2181
[zk: localhost:2181(CONNECTED) 0] addauth digest user:pass
""")

print("\n=== Basic Commands ===")
print("""
# List
ls /
ls /zookeeper
ls /brokers

# Read
get /myapp
get -s /myapp  # With stat
get -w /myapp  # With watch

# Create
create /myapp "data"
create -e /myapp "ephemeral"
create -s /myapp "sequential"
create -e -s /myapp "ephemeral_seq"

# Delete
delete /myapp
deleteall /myapp  # Recursive

# Stat
stat /myapp

# Set
set /myapp "new_data"

# Version
set /myapp "data" 1  # With version
""")

print("\n=== Advanced Commands ===")
print("""
# Watch
get -w /myapp
ls -w /myapp

# ACL
getAcl /myapp
setAcl /myapp world:anyone:cdrwa

# Quota
setquota -n 100000 /myapp
listquota /myapp
delquota /myapp

# Sync
sync /myapp

# History
history

# Redo
redo 5

# Close
close
quit
""")

print("\n=== Four-Letter Words ===")
print("""
Administrative commands:
  mntr    # Health metrics
  stat    # Server stats
  ruok    # Are you OK?
  dump    # Dump sessions/znodes
  conf    # Configuration
  envi    # Environment
  srvr    # Server stats
  cons    # Connections
  crst    # Reset stats
  wchs    # Watch summary
  wchc    # Watch by connection
  wchp    # Watch by path
  dirs    # Directory size
  isro    # Read-only mode
  srst    # Reset server stats

Usage:
  echo mntr | nc localhost 2181
  echo stat | nc localhost 2181
  echo ruok | nc localhost 2181
  echo cons | nc localhost 2181
""")

print("\n=== Example Session ===")
print("""
$ zkCli.sh
[zk: localhost:2181(CONNECTED) 0] ls /
[zookeeper]

[zk: localhost:2181(CONNECTED) 1] create /myapp "Hello"
Created /myapp

[zk: localhost:2181(CONNECTED) 2] get /myapp
Hello
cZxid = 0x5
ctime = Mon Jan 15 10:00:00 UTC 2025
mZxid = 0x5
mtime = Mon Jan 15 10:00:00 UTC 2025
pZxid = 0x5
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 5
numChildren = 0

[zk: localhost:2181(CONNECTED) 3] create -e /myapp/lock "owner"
Created /myapp/lock

[zk: localhost:2181(CONNECTED) 4] ls /myapp
[lock]

[zk: localhost:2181(CONNECTED) 5] delete /myapp
Node not empty: /myapp

[zk: localhost:2181(CONNECTED) 6] deleteall /myapp
Deleted /myapp

[zk: localhost:2181(CONNECTED) 7] quit
""")

print("\n=== Monitoring ===")
print("""
# Health check
$ echo ruok | nc localhost 2181
imok

# Metrics
$ echo mntr | nc localhost 2181
zk_version	3.8.0
zk_avg_latency	0
zk_max_latency	0
zk_min_latency	0
zk_packets_received	10
zk_packets_sent	9
zk_num_alive_connections	1
zk_outstanding_requests	0
zk_server_state	leader
zk_znode_count	5
zk_watch_count	0
zk_ephemerals_count	0
zk_approximate_data_size	40
zk_open_file_descriptor_count	30
zk_max_file_descriptor_count	1024

# Connections
$ echo cons | nc localhost 2181
 /127.0.0.1:52345[0](queued=0,recved=1,sent=0)
""")

print("\n=== Best Practices ===")
print("""
1. Use -s for stat
2. Use -w for watch
3. Use deleteall for recursive
4. Check stat before write
5. Use version for CAS
6. Monitor with mntr
7. Clean up old znodes
8. Document
""")

💻 Client API

🎯 Programming ZooKeeper

ZooKeeper has client libraries in Java, Python, Go, C, and more. Apache Curator and Kazoo are popular high-level libraries.

Client API

Java: Native + Curator. Python: Kazoo. Go: go-zookeeper. C: Native. Node.js: node-zookeeper. Curator: Recommended.

💻 Java💻 Python💻 Go💻 Curator

# Client API
print("=== Java (Native) ===")
print("""
import org.apache.zookeeper.*;
import org.apache.zookeeper.data.Stat;

public class ZKClient implements Watcher {
    
    private ZooKeeper zk;
    
    public ZKClient(String hosts) throws Exception {
        this.zk = new ZooKeeper(hosts, 3000, this);
    }
    
    @Override
    public void process(WatchedEvent event) {
        System.out.println("Event: " + event);
    }
    
    public void create(String path, byte[] data) throws Exception {
        zk.create(path, data,
                  ZooDefs.Ids.OPEN_ACL_UNSAFE,
                  CreateMode.PERSISTENT);
    }
    
    public byte[] read(String path) throws Exception {
        Stat stat = new Stat();
        return zk.getData(path, this, stat);
    }
    
    public void delete(String path) throws Exception {
        zk.delete(path, -1);
    }
    
    public static void main(String[] args) throws Exception {
        ZKClient client = new ZKClient("localhost:2181");
        client.create("/myapp", "Hello".getBytes());
        byte[] data = client.read("/myapp");
        System.out.println(new String(data));
    }
}
""")

print("\n=== Java (Curator) ===")
print("""
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.curator.retry.ExponentialBackoffRetry;
import org.apache.curator.framework.recipes.leader.LeaderLatch;
import org.apache.curator.framework.recipes.locks.InterProcessMutex;

public class CuratorExample {
    
    public static void main(String[] args) throws Exception {
        CuratorFramework client = CuratorFrameworkFactory.newClient(
            "localhost:2181",
            new ExponentialBackoffRetry(1000, 3)
        );
        client.start();
        
        // Create
        client.create().forPath("/myapp", "Hello".getBytes());
        
        // Read
        byte[] data = client.getData().forPath("/myapp");
        System.out.println(new String(data));
        
        // Watch
        client.getData().watched().forPath("/myapp");
        
        // Leader election
        LeaderLatch latch = new LeaderLatch(client, "/election");
        latch.start();
        if (latch.hasLeadership()) {
            System.out.println("I am the leader");
        }
        
        // Distributed lock
        InterProcessMutex lock = new InterProcessMutex(client, "/lock");
        if (lock.acquire(10, TimeUnit.SECONDS)) {
            try {
                // Critical section
            } finally {
                lock.release();
            }
        }
        
        // Cleanup
        client.close();
    }
}
""")

print("\n=== Python (Kazoo) ===")
print("""
from kazoo.client import KazooClient
from kazoo.recipe.lock import Lock
from kazoo.recipe.election import Election
from kazoo.recipe.barrier import Barrier

# Connect
zk = KazooClient(hosts='localhost:2181')
zk.start()

# Create
zk.create("/myapp", b"Hello")

# Read
data, stat = zk.get("/myapp")
print(f"Data: {data}, Version: {stat.version}")

# Update with version
zk.set("/myapp", b"New", version=stat.version)

# Watch
def watch(event):
    print(f"Event: {event}")
    zk.get("/myapp", watch=watch)

zk.get("/myapp", watch=watch)

# Children
children = zk.get_children("/myapp")
print(f"Children: {children}")

# Delete
zk.delete("/myapp")

# Lock
lock = Lock(zk, "/lock")
with lock:
    print("In critical section")

# Election
def leader_func():
    print("I am leader")

election = Election(zk, "/election", "candidate")
election.run(leader_func)

# Barrier
barrier = Barrier(zk, "/barrier", 3)
barrier.wait()

# Disconnect
zk.stop()
zk.close()
""")

print("\n=== Go ===")
print("""
package main

import (
    "fmt"
    "time"
    "github.com/go-zookeeper/zk"
)

func main() {
    // Connect
    conn, _, err := zk.Connect([]string{"localhost:2181"}, time.Second)
    if err != nil {
        panic(err)
    }
    defer conn.Close()
    
    // Create
    _, err = conn.Create("/myapp", []byte("Hello"),
                         0, zk.WorldACL(zk.PermAll))
    
    // Read
    data, stat, err := conn.Get("/myapp")
    fmt.Printf("Data: %s, Version: %d\\n", data, stat.Version)
    
    // Watch
    _, _, events, _ := conn.GetW("/myapp")
    go func() {
        for event := range events {
            fmt.Printf("Event: %v\\n", event)
        }
    }()
    
    // Children
    children, _, _ := conn.Children("/myapp")
    fmt.Printf("Children: %v\\n", children)
    
    // Delete
    conn.Delete("/myapp", -1)
}
""")

print("\n=== Comparison ===")
print(f"{'Library':<20} {'Language':<15} {'Pros':<25}")
print("-" * 60)
print(f"{'Native':<20} {'Java':<15} {'Full API':<25}")
print(f"{'Curator':<20} {'Java':<15} {'Recipes, tested':<25}")
print(f"{'Kazoo':<20} {'Python':<15} {'Clean API':<25}")
print(f"{'go-zookeeper':<20} {'Go':<15} {'Native':<25}")
print(f"{'node-zookeeper':<20} {'Node.js':<15} {'Popular':<25}")

print("\n=== Best Practices ===")
print("""
1. Use Curator (Java)
   - Recipes
   - Tested
   - Managed

2. Use Kazoo (Python)
   - Clean API
   - Recipes

3. Handle state changes
   - LOST
   - SUSPENDED
   - CONNECTED

4. Set timeouts
   - Reasonable
   - Handle

5. Close properly
   - stop()
   - close()

6. Use ephemeral
   - Auto cleanup

7. Watch pattern
   - Re-register
   - Or use library

8. Log
   - Events
   - Errors
   - Metrics
""")

🔧 Operations

🎯 Running ZooKeeper

Operating ZooKeeper in production requires planning: ensemble size, hardware, monitoring, backup, and tuning.

Operations

Ensemble: 3 or 5. Hardware: Fast disk. Monitoring: mntr, metrics. Backup: Snapshots. Tuning: JVM, disk. Security: SASL, ACLs.

🔧 Ensemble🔧 Monitoring🔧 Backup🔧 Tuning

# Operations
print("=== Ensemble Setup ===")
print("""
Sizes:
  - 3 nodes: Production (tolerate 1)
  - 5 nodes: Production (tolerate 2)
  - 7 nodes: Large (tolerate 3)

Rules:
  - Odd number (quorum)
  - Same hardware
  - Separate failure domains
  - Low latency network

Configuration:
  # zoo.cfg
  tickTime=2000
  initLimit=10
  syncLimit=5
  dataDir=/var/lib/zookeeper
  dataLogDir=/var/log/zookeeper
  clientPort=2181
  maxClientCnxns=60
  
  # Ensemble
  server.1=zk1:2888:3888
  server.2=zk2:2888:3888
  server.3=zk3:2888:3888
  
  # Advanced
  autopurge.snapRetainCount=3
  autopurge.purgeInterval=1
  maxSessionTimeout=40000
  minSessionTimeout=4000
""")

print("\n=== myid File ===")
print("""
Each server needs myid:
  $ echo "1" > /var/lib/zookeeper/myid
  $ echo "2" > /var/lib/zookeeper/myid
  $ echo "3" > /var/lib/zookeeper/myid
""")

print("\n=== Hardware ===")
print("""
CPU:
  - 2+ cores
  - Not CPU-bound

RAM:
  - 4-8 GB
  - Heap: 2-4 GB
  - Data fits in memory

Disk:
  - Fast SSD
  - Dedicated log disk
  - Low latency critical

Network:
  - Low latency
  - Redundant
  - Isolated (if possible)

Example:
  - 4 cores, 8 GB RAM
  - 2 × SSD (data + log)
  - 10 Gbps network
""")

print("\n=== JVM Tuning ===")
print("""
# java.env
export JVMFLAGS="-Xms2g -Xmx2g -XX:+UseG1GC"

Recommendations:
  - Xms = Xmx
  - G1GC
  - Heap: 2-4 GB
  - Avoid swapping

Why:
  - Data in memory
  - Predictable
  - Low latency
""")

print("\n=== Monitoring ===")
print("""
Key metrics:
  - zk_avg_latency
  - zk_max_latency
  - zk_znode_count
  - zk_watch_count
  - zk_ephemerals_count
  - zk_num_alive_connections
  - zk_outstanding_requests
  - zk_open_file_descriptor_count
  - zk_server_state
  - zk_followers
  - zk_synced_followers

Tools:
  - Prometheus
  - Grafana
  - Datadog
  - New Relic
  - Zabbix

Commands:
  echo mntr | nc localhost 2181
  echo stat | nc localhost 2181
  echo srvr | nc localhost 2181
  echo cons | nc localhost 2181
""")

print("\n=== Logging ===")
print("""
Logs:
  - zookeeper.out
  - log4j.properties
  - /var/log/zookeeper/

Levels:
  - INFO (default)
  - DEBUG (troubleshooting)
  - WARN
  - ERROR

Content:
  - Leader election
  - Session events
  - Errors
  - Warnings
""")

print("\n=== Backup ===")
print("""
What to backup:
  - dataDir (snapshots)
  - dataLogDir (transaction logs)
  - zoo.cfg
  - myid
  - ACLs

Frequency:
  - Daily snapshots
  - Continuous txn logs

Restore:
  1. Stop ZooKeeper
  2. Restore files
  3. Start ZooKeeper

Best:
  - Automate
  - Test restore
  - Offsite
""")

print("\n=== Security ===")
print("""
1. AUTHENTICATION
   - SASL/Kerberos
   - Digest
   - x509
   
   requireClientAuthScheme=sasl

2. AUTHORIZATION
   - ACLs per znode
   - world, auth, digest, ip, sasl
   
   setAcl /myapp digest:user:pass:cdrwa

3. ENCRYPTION
   - TLS for client port
   - TLS for ensemble
   
   secureClientPort=2281
   serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
   ssl.keyStore.location=...
   ssl.trustStore.location=...

4. NETWORK
   - Firewall
   - VPC
   - Private network

5. AUDIT
   - Log access
   - Monitor
""")

print("\n=== Upgrade ===")
print("""
Rolling upgrade:
  1. Backup
  2. Stop one follower
  3. Upgrade
  4. Start
  5. Verify
  6. Repeat for others
  7. Leader last
  8. Force election if needed

Or:
  - Add new servers (upgraded)
  - Remove old
  - Zero downtime
""")

print("\n=== Best Practices ===")
print("""
1. 3 or 5 servers
2. Separate failure domains
3. Fast disk (SSD)
4. Ample RAM
5. Low latency network
6. Monitor
7. Backup
8. Secure
9. Upgrade carefully
10. Document
11. Test failover
12. Capacity planning
""")

💼 Use Cases

🎯 Who Uses ZooKeeper

ZooKeeper is the coordination backbone of many distributed systems. Here are the most important use cases.

Use Cases

Kafka: Metadata, controller. HBase: Region assignment. Hadoop: HA. Solr: Cluster state. Dubbo: Service discovery. Custom: Apps.

💼 Kafka💼 HBase💼 Hadoop💼 Microservices

# Use Cases
print("=== Kafka ===")
print("""
Kafka:
  - Cluster membership
  - Controller election
  - Topic configuration
  - ACLs
  - Quotas
  - Partition assignment

Paths:
  /brokers/ids         # Broker registration
  /brokers/topics      # Topic metadata
  /controller          # Controller leader
  /config              # Configuration
  /admin               # Admin operations

Note:
  - Kafka 2.8+ has KRaft
  - Kafka 4.0 removed ZooKeeper
  - Legacy still common
""")

print("\n=== HBase ===")
print("""
HBase:
  - Master election
  - RegionServer membership
  - Region assignment
  - Table metadata
  - Split/merge coordination

Paths:
  /hbase/master        # Active master
  /hbase/rs            # RegionServers
  /hbase/meta-region-server
  /hbase/table         # Table state
""")

print("\n=== Hadoop HA ===")
print("""
Hadoop HA:
  - NameNode election
  - Active/Standby
  - Failover
  - Split-brain prevention

Paths:
  /hadoop-ha/mycluster/ActiveStandbyElectorLock
  /hadoop-ha/mycluster/ActiveBreadCrumb
""")

print("\n=== Solr ===")
print("""
Solr:
  - Cluster state
  - Collection metadata
  - Shard assignment
  - Leader election per shard

Paths:
  /solr/live_nodes     # Live nodes
  /solr/collections    # Collections
  /solr/aliases        # Aliases
""")

print("\n=== Microservices ===")
print("""
Service Discovery:
  - Register services
  - Discover instances
  - Health check
  - Load balancing

Paths:
  /services/{name}/instances/{id}
  /services/{name}/config

Libraries:
  - Apache Curator
  - Netflix Eureka (own)
  - Consul (own)
  - etcd (own)
""")

print("\n=== Distributed Systems ===")
print("""
Other uses:
  - Dubbo (RPC framework)
  - Mesos (cluster manager)
  - Storm (stream processing)
  - Flink (HA)
  - Neo4j (cluster)
  - Akka (cluster)
  - Custom apps
""")

print("\n=== Custom Applications ===")
print("""
Patterns:
  - Configuration management
  - Distributed locks
  - Leader election
  - Group membership
  - Distributed queues
  - Barriers
  - Counters
  - Caches

Examples:
  - Job scheduler
  - Microservice coordination
  - Distributed cron
  - Cache invalidation
  - Feature flags
""")

print("\n=== Comparison ===")
print(f"{'System':<20} {'ZooKeeper Use':<40}")
print("-" * 60)
print(f"{'Kafka':<20} {'Metadata, controller':<40}")
print(f"{'HBase':<20} {'Master, region assignment':<40}")
print(f"{'Hadoop':<20} {'NameNode HA':<40}")
print(f"{'Solr':<20} {'Cluster state':<40}")
print(f"{'Dubbo':<20} {'Service discovery':<40}")
print(f"{'Storm':<20} {'Cluster coordination':<40}")
print(f"{'Flink':<20} {'JobManager HA':<40}")

print("\n=== Modern Alternatives ===")
print("""
Alternatives:
  - etcd (Kubernetes)
  - Consul (HashiCorp)
  - Raft (built-in)
  - KRaft (Kafka)
  - Custom Raft/Paxos

ZooKeeper still:
  - Mature
  - Battle-tested
  - Widely used
  - Proven

Use when:
  - Existing investment
  - Need recipes
  - Proven track record
""")

❓ FAQ

🎯 Frequently Asked Questions

Answers to common questions about Apache ZooKeeper.

What is Apache ZooKeeper?

Apache ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. It's used by Kafka, HBase, Hadoop, and many distributed systems for coordination.

What is a znode?

A znode is a node in ZooKeeper's hierarchical namespace, similar to a file or directory in a filesystem. Znodes store data, have a path, and can be persistent, ephemeral, sequential, or a combination.

What is a watch in ZooKeeper?

A watch is a one-time trigger that clients set on znodes to receive notifications when the znode changes (created, deleted, data changed, children changed). Watches enable clients to react to changes without polling.

How does ZooKeeper achieve consensus?

ZooKeeper uses the ZAB (ZooKeeper Atomic Broadcast) protocol, a variant of Paxos, to achieve consensus among ensemble members. A leader is elected, and all writes go through the leader, which broadcasts to followers.

Does Kafka still need ZooKeeper?

No. Since Kafka 2.8, KRaft mode allows Kafka to run without ZooKeeper. Kafka 3.x supports both; Kafka 4.0 (2025) removed ZooKeeper entirely. However, many existing deployments still use ZooKeeper.

How many servers should be in a ZooKeeper ensemble?

Use an odd number: 3 for production (tolerates 1 failure), 5 for larger scale (tolerates 2 failures), 7 for very large (tolerates 3 failures). Odd numbers ensure clean quorum. 1 is only for development.

What is the difference between ephemeral and persistent znodes?

Persistent znodes survive session end until explicitly deleted. Ephemeral znodes are deleted automatically when the session ends. Ephemeral is used for locks, leader election, and membership.

Can I store large data in ZooKeeper?

No. Znodes have a 1 MB limit (recommended <1 KB). ZooKeeper is for metadata and coordination, not bulk data. Use object storage (S3, HDFS) for large data.

What is the difference between ZooKeeper and etcd?

Both provide distributed coordination, but etcd is more modern, uses the Raft protocol, and is the standard for Kubernetes. ZooKeeper uses ZAB and is more mature with wider adoption in Hadoop ecosystem. etcd is simpler; ZooKeeper has richer recipes (via Curator).

How do I monitor ZooKeeper?

Use the four-letter words (mntr, stat, srvr, cons) to get metrics. Integrate with Prometheus, Grafana, Datadog, or New Relic. Key metrics include latency, znode count, watch count, and connection count.

# Quick Reference
print("=== Quick Answers ===")
print("""
Q: Is ZooKeeper a database?
A: No — it's a coordination service.

Q: Does Kafka need it?
A: No — KRaft since 2.8/3.0.

Q: How many nodes?
A: Odd — 3 or 5.

Q: How much data?
A: Metadata only — <1KB per znode.

Q: What's a znode?
A: Node in the tree — like a file.

Q: What's a watch?
A: One-time notification.

Q: What's ZAB?
A: Consensus protocol (Paxos variant).

Q: Java or Python?
A: Java (Curator) — most common.
   Python (Kazoo) — also popular.

Q: Alternative?
A: etcd, Consul (for new projects).

Q: Still relevant?
A: Yes — Hadoop ecosystem uses it.
""")

🚀 Future

🎯 Where ZooKeeper Is Going

ZooKeeper remains critical in the Hadoop ecosystem but faces competition from etcd, Consul, and built-in consensus (KRaft). The future is niche but stable.

Future Directions

Stable: Mature. Niche: Hadoop. KRaft: Replacing in Kafka. etcd: Growing. Cloud: Managed. Open: Apache.

🚀 Stable🚀 Niche🚀 KRaft🚀 etcd

# Future of ZooKeeper
print("=== 1. Stable but Niche ===")
print("""
Trend: Mature, niche

Reality:
  - Mature (since 2008)
  - Stable API
  - Widely used
  - Apache project

Niche:
  - Hadoop ecosystem
  - Legacy Kafka
  - HBase
  - Solr

New projects:
  - Prefer etcd
  - Or Consul
  - Or built-in consensus
""")

print("\n=== 2. Kafka KRaft ===")
print("""
Trend: KRaft replacing ZooKeeper

Timeline:
  2.8 (2021): KRaft early access
  3.0 (2021): Production-ready
  3.3 (2022): KRaft default for new
  3.5 (2023): KRaft mature
  4.0 (2025): ZooKeeper removed

Impact:
  - Fewer new Kafka+ZK deployments
  - Existing still on ZK
  - Migration ongoing

Future:
  - KRaft standard
  - ZooKeeper for legacy
""")

print("\n=== 3. etcd/Consul ===")
print("""
Trend: Modern alternatives

etcd:
  - Raft
  - Kubernetes standard
  - Simple
  - gRPC API

Consul:
  - Service mesh
  - Health checks
  - Multi-datacenter
  - HashiCorp

Comparison:
  - etcd: K8s-native
  - Consul: Service mesh
  - ZooKeeper: Hadoop ecosystem

Future:
  - etcd dominates K8s
  - Consul for service mesh
  - ZooKeeper for Hadoop
""")

print("\n=== 4. Cloud Managed ===")
print("""
Trend: Managed services

Options:
  - AWS MSK (Kafka)
  - Confluent Cloud (no ZK)
  - Aiven
  - Instaclustr
  - Cloud-native (self-managed)

Benefits:
  - No ops
  - Scalable
  - Managed

Future:
  - Cloud default
  - Fewer self-managed
""")

print("\n=== 5. New Consensus ===")
print("""
Trend: Built-in consensus

Raft:
  - Simpler than Paxos
  - Many implementations
  - Built into systems

Examples:
  - KRaft (Kafka)
  - etcd
  - Consul
  - CockroachDB
  - TiDB

Future:
  - More built-in
  - Fewer external
  - ZooKeeper less needed
""")

print("\n=== 6. Legacy Support ===")
print("""
Trend: Long tail

Reality:
  - Many systems use ZK
  - Millions of deployments
  - Years of code
  - Critical infrastructure

Support:
  - Apache continues
  - Security updates
  - Bug fixes
  - Not much new features

Future:
  - Maintained
  - Legacy
  - Stable
""")

# TIMELINE
print("\n=== Timeline ===")
print("""
Past:
  2006: ZooKeeper at Yahoo
  2008: Apache top-level
  2010: Kafka uses ZK
  2013: Curator
  2018: Hadoop uses ZK

Present:
  2024: Still widely used
  2025: Kafka 4.0 removes ZK

Future:
  2030: Niche?
  2035: Legacy?
  2040: Unknown
""")

print("\n=== Final Thoughts ===")
print("""
ZooKeeper is:
  - Mature
  - Reliable
  - Widely used
  - Apache
  - Coordination

The future is:
  - Stable, niche
  - KRaft replaces in Kafka
  - etcd grows in K8s
  - Cloud-managed
  - Legacy support

"Coordination is fundamental.
 ZooKeeper is proven."

Keep exploring. Keep building.
""")