π§ Hash Function Fundamentals
π― What is a Cryptographic Hash Function?
A cryptographic hash function is a mathematical algorithm that takes an input (or "message") and produces a fixed-size output (hash). It is deterministic, one-way (preimage resistant), collision-resistant, and exhibits the avalanche effect. Hash functions are fundamental to cryptography, blockchain, digital signatures, password storage, and data integrity.
Why Hashing Matters
Integrity: Detect changes. Authentication: Verify identity. Signatures: Digital signing. Blockchain: Block linking. Passwords: Secure storage. Merkle: Efficient verification.
# Cryptographic Hash Functions β the big picture
# One-way functions for security and integrity.
print("""
=== Cryptographic Hash Functions ===
What it is:
- Input β fixed-size output
- One-way
- Deterministic
- Collision-resistant
- Avalanche effect
Why it matters:
- Data integrity
- Authentication
- Digital signatures
- Blockchain
- Password storage
- Merkle trees
- HMAC
Key properties:
- Preimage resistance
- Second preimage resistance
- Collision resistance
- Avalanche effect
- Deterministic
- Fixed output size
- Fast
Common algorithms:
- MD5 (broken)
- SHA-1 (broken)
- SHA-2 (SHA-256, SHA-512)
- SHA-3 (Keccak)
- BLAKE2
- BLAKE3
- RIPEMD-160
Applications:
- Blockchain
- Digital signatures
- Password hashing
- File integrity
- Git
- TLS/SSL
- Cryptocurrency
- Merkle trees
""")
print("=== Historical Context ===")
print("""
1989: MD2 (Rivest)
1991: MD5 (Rivest)
1993: SHA-0
1995: SHA-1
1998: RIPEMD-160
2001: SHA-2 (SHA-256, SHA-512)
2004: MD5 collisions
2005: SHA-1 collisions
2008: BLAKE
2012: Keccak (SHA-3)
2015: BLAKE2
2020: BLAKE3
2025: Post-quantum hashing
""")
print("=== Why It's Hard ===")
print("""
1. COLLISION RESISTANCE
- Birthday paradox
- 2^(n/2) attacks
- MD5, SHA-1 broken
2. LENGTH EXTENSION
- Merkle-DamgΓ₯rd
- SHA-2 vulnerable
- SHA-3 resistant
3. QUANTUM
- Grover's algorithm
- 2^(n/2) speedup
- Post-quantum needed
4. PERFORMANCE
- Fast
- Hardware
- Parallel
5. PASSWORDS
- MD5 too fast
- Need slow hashing
- bcrypt, Argon2
""")π― Why Hashing Matters
π― The Value of Hash Functions
Hash functions are the workhorses of modern cryptography. They enable data integrity, authentication, digital signatures, and blockchain, among many other applications.
Key Applications
Integrity: Detect tampering. Authentication: Verify. Signatures: Sign. Blockchain: Link. Passwords: Store. Merkle: Verify.
# Why Hashing Matters
print("=== 1. Data Integrity ===")
print("""
Integrity:
- Hash file
- Verify
- Detect changes
Example:
File β SHA-256
β 2cf24dba...
β Verify
""")
print("\n=== 2. Authentication ===")
print("""
Authentication:
- Challenge-response
- Hash password
- Verify
Example:
Password β Hash
β Compare
""")
print("\n=== 3. Digital Signatures ===")
print("""
Signatures:
- Hash message
- Sign hash
- Verify
Example:
Message β SHA-256
β Sign
β Signature
""")
print("\n=== 4. Blockchain ===")
print("""
Blockchain:
- Block hash
- Previous hash
- Chain
Example:
Block β SHA-256
β Link
""")
print("\n=== 5. Password Storage ===")
print("""
Passwords:
- Hash
- Salt
- Slow
Example:
Password β bcrypt
β Hash
""")
print("\n=== 6. Merkle Trees ===")
print("""
Merkle:
- Hash tree
- Root
- Verify
Example:
Transactions β Merkle root
""")
print("\n=== 7. HMAC ===")
print("""
HMAC:
- Hash + key
- Authentication
- Integrity
Example:
HMAC(key, message)
""")
print("\n=== 8. Git ===")
print("""
Git:
- SHA-1
- Commit hash
- Content-addressed
Example:
Commit β SHA-1
β Hash
""")
print("\n=== 9. TLS/SSL ===")
print("""
TLS:
- Certificate
- Signature
- Hash
Example:
Certificate β SHA-256
β Sign
""")
print("\n=== 10. Cryptocurrency ===")
print("""
Crypto:
- Mining
- Addresses
- Transactions
Example:
Bitcoin β SHA-256
""")π Properties
π― Hash Function Properties
A cryptographic hash function must satisfy several properties to be secure: preimage resistance, second preimage resistance, collision resistance, and the avalanche effect.
Properties
Preimage: Cannot reverse. Second preimage: Cannot find alternative. Collision: Cannot find two. Avalanche: Small change. Deterministic: Same input. Fixed: Fixed size.
# Properties
print("=== 1. Preimage Resistance ===")
print("""
Preimage:
- Given hash h
- Cannot find m such that H(m) = h
- One-way
- 2^n effort
Example:
h = 2cf24dba...
β Cannot find m
""")
print("\n=== 2. Second Preimage Resistance ===")
print("""
Second preimage:
- Given m1
- Cannot find m2 β m1
- H(m1) = H(m2)
- 2^n effort
Example:
m1 = "hello"
β Cannot find m2
""")
print("\n=== 3. Collision Resistance ===")
print("""
Collision:
- Cannot find m1 β m2
- H(m1) = H(m2)
- 2^(n/2) effort (birthday)
Example:
MD5: broken
SHA-1: broken
SHA-256: secure
""")
print("\n=== 4. Avalanche Effect ===")
print("""
Avalanche:
- Small change
- Large difference
- 50% bits
Example:
"hello" β 2cf24dba...
"hellp" β 5d41402a...
β Completely different
""")
print("\n=== 5. Deterministic ===")
print("""
Deterministic:
- Same input
- Same output
- Always
Example:
SHA-256("hello")
β 2cf24dba...
β Always
""")
print("\n=== 6. Fixed Output ===")
print("""
Fixed:
- Fixed size
- Regardless of input
- 256 bits
Example:
"hello" β 256 bits
"hello world" β 256 bits
""")
print("\n=== 7. Fast ===")
print("""
Fast:
- Efficient
- Hardware
- Parallel
Example:
SHA-256
β GB/s
""")
print("\n=== 8. Birthday Paradox ===")
print("""
Birthday:
- Collision probability
- 2^(n/2)
- n = hash size
Example:
SHA-256: 2^128
β Secure
""")
print("\n=== 9. Length Extension ===")
print("""
Length extension:
- Merkle-DamgΓ₯rd
- SHA-2 vulnerable
- SHA-3 resistant
Example:
SHA-256
β Vulnerable
""")
print("\n=== 10. Python Properties ===")
print("""
import hashlib
def verify_properties():
\"\"\"Demonstrate properties.\"\"\"
# Deterministic
h1 = hashlib.sha256(b"hello").hexdigest()
h2 = hashlib.sha256(b"hello").hexdigest()
assert h1 == h2
# Avalanche
h3 = hashlib.sha256(b"hellp").hexdigest()
assert h1 != h3
# Fixed size
h4 = hashlib.sha256(b"a" * 1000).hexdigest()
assert len(h1) == len(h4) == 64
return True
# verify_properties()
""")π SHA-2 (SHA-256)
π― SHA-256
SHA-256 is part of the SHA-2 family, designed by the NSA and published by NIST in 2001. It produces a 256-bit hash and is widely used in blockchain (Bitcoin), TLS, and digital signatures.
SHA-256
Output: 256 bits. Block: 512 bits. Rounds: 64. Construction: Merkle-DamgΓ₯rd. Bitcoin: Uses. Secure: Yes.
# SHA-2 (SHA-256)
print("=== 1. Overview ===")
print("""
SHA-256:
- SHA-2 family
- NSA/NIST
- 2001
- 256-bit output
Parameters:
- Block size: 512 bits
- Word size: 32 bits
- Rounds: 64
- Output: 256 bits
""")
print("\n=== 2. Algorithm ===")
print("""
Steps:
1. Padding
2. Parse into blocks
3. Initialize hash values
4. Process blocks
5. Output
""")
print("\n=== 3. Constants ===")
print("""
Constants:
- 64 round constants (K)
- 8 initial hash values (H)
- Cube roots of primes
- Square roots of primes
Example:
H0 = 6a09e667
H1 = bb67ae85
...
""")
print("\n=== 4. Operations ===")
print("""
Operations:
- Right rotate
- Right shift
- XOR
- AND
- NOT
- Addition mod 2^32
Functions:
- Ch(x, y, z)
- Maj(x, y, z)
- Ξ£0, Ξ£1, Ο0, Ο1
""")
print("\n=== 5. Python SHA-256 ===")
print("""
import hashlib
# Hash
h = hashlib.sha256(b"hello").hexdigest()
print(h)
# 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824
# Incremental
h = hashlib.sha256()
h.update(b"hello")
h.update(b" world")
print(h.hexdigest())
""")
print("\n=== 6. SHA-512 ===")
print("""
SHA-512:
- SHA-2 family
- 512-bit output
- 64-bit words
- 80 rounds
Example:
import hashlib
hashlib.sha512(b"hello").hexdigest()
""")
print("\n=== 7. SHA-224 ===")
print("""
SHA-224:
- SHA-2 family
- 224-bit
- Truncated SHA-256
Example:
hashlib.sha224(b"hello").hexdigest()
""")
print("\n=== 8. SHA-384 ===")
print("""
SHA-384:
- SHA-2 family
- 384-bit
- Truncated SHA-512
Example:
hashlib.sha384(b"hello").hexdigest()
""")
print("\n=== 9. Security ===")
print("""
Security:
- Preimage: 2^256
- Collision: 2^128
- Secure
- Bitcoin
Status:
- No practical attacks
- Recommended
""")
print("\n=== 10. Use Cases ===")
print("""
Use cases:
- Bitcoin
- TLS/SSL
- Digital signatures
- File integrity
- HMAC
- Password hashing (with salt)
- Merkle trees
""")π§ SHA-3 (Keccak)
π― SHA-3
SHA-3 (Keccak) is the latest member of the Secure Hash Algorithm family, standardized by NIST in 2015. It uses a sponge construction, providing resistance to length extension attacks.
SHA-3
Keccak: Sponge. Output: 224-512. Rounds: 24. State: 1600 bits. Ethereum: Keccak-256. Secure: Yes.
# SHA-3 (Keccak)
print("=== 1. Overview ===")
print("""
SHA-3:
- Keccak
- NIST competition
- 2015
- Sponge construction
Parameters:
- State: 1600 bits
- Rate + capacity = 1600
- Rounds: 24
- Output: 224, 256, 384, 512
""")
print("\n=== 2. Sponge Construction ===")
print("""
Sponge:
- Absorb phase
- Squeeze phase
- State
Steps:
1. Initialize state
2. Absorb input
3. Squeeze output
""")
print("\n=== 3. Keccak-f ===")
print("""
Keccak-f:
- 24 rounds
- 5 steps:
- ΞΈ (theta)
- Ο (rho)
- Ο (pi)
- Ο (chi)
- ΞΉ (iota)
State:
- 5x5x64 bits
- 1600 bits
""")
print("\n=== 4. Python SHA-3 ===")
print("""
import hashlib
# SHA-3
h = hashlib.sha3_256(b"hello").hexdigest()
print(h)
# SHAKE
h = hashlib.shake_128(b"hello").hexdigest(16)
print(h)
""")
print("\n=== 5. Keccak-256 ===")
print("""
Keccak-256:
- Ethereum
- Original Keccak
- Not NIST SHA-3
- Padding differs
Example:
from Crypto.Hash import keccak
k = keccak.new(digest_bits=256)
k.update(b"hello")
print(k.hexdigest())
""")
print("\n=== 6. SHAKE ===")
print("""
SHAKE:
- Extendable Output Function (XOF)
- Variable length
- SHAKE128, SHAKE256
Example:
hashlib.shake_128(b"hello").hexdigest(32)
""")
print("\n=== 7. Security ===")
print("""
Security:
- Preimage: 2^256
- Collision: 2^128
- Secure
- No attacks
Status:
- Recommended
- Post-quantum
""")
print("\n=== 8. Advantages ===")
print("""
Advantages:
- Sponge construction
- Length extension resistant
- Flexible output
- Different design
- Backup for SHA-2
""")
print("\n=== 9. Use Cases ===")
print("""
Use cases:
- Ethereum (Keccak-256)
- NIST standards
- SHAKE (XOF)
- Post-quantum
- Backup
""")
print("\n=== 10. Python Examples ===")
print("""
import hashlib
# SHA-3 variants
print(hashlib.sha3_224(b"hello").hexdigest())
print(hashlib.sha3_256(b"hello").hexdigest())
print(hashlib.sha3_384(b"hello").hexdigest())
print(hashlib.sha3_512(b"hello").hexdigest())
# SHAKE
print(hashlib.shake_128(b"hello").hexdigest(16))
print(hashlib.shake_256(b"hello").hexdigest(32))
""")β οΈ MD5 & RIPEMD
π― Legacy Hash Functions
MD5 and RIPEMD-160 are older hash functions. MD5 is broken (collisions), while RIPEMD-160 is still considered secure but less common.
Legacy
MD5: Broken. RIPEMD: Secure. 128-bit: MD5. 160-bit: RIPEMD. Bitcoin: RIPEMD. Avoid: MD5.
# MD5 & RIPEMD
print("=== 1. MD5 ===")
print("""
MD5:
- Rivest
- 1991
- 128-bit
- Broken
Output:
- 128 bits
- 32 hex chars
Example:
hashlib.md5(b"hello").hexdigest()
β 5d41402abc4b2a76b9719d911017c592
""")
print("\n=== 2. MD5 Collisions ===")
print("""
Collisions:
- 2004: practical
- 2008: rogue CA
- 2012: Flame malware
- Broken
Example:
Two different files
β Same MD5
""")
print("\n=== 3. RIPEMD-160 ===")
print("""
RIPEMD-160:
- 1996
- 160-bit
- Secure
- Bitcoin
Output:
- 160 bits
- 40 hex chars
Example:
hashlib.new('ripemd160', b"hello").hexdigest()
""")
print("\n=== 4. RIPEMD Family ===")
print("""
RIPEMD:
- RIPEMD-128
- RIPEMD-160
- RIPEMD-256
- RIPEMD-320
Most common:
- RIPEMD-160
""")
print("\n=== 5. Bitcoin Addresses ===")
print("""
Bitcoin:
- SHA-256
- RIPEMD-160
- Base58Check
Example:
Public key β SHA-256
β RIPEMD-160
β Address
""")
print("\n=== 6. Python MD5 ===")
print("""
import hashlib
# MD5
h = hashlib.md5(b"hello").hexdigest()
print(h)
# 5d41402abc4b2a76b9719d911017c592
""")
print("\n=== 7. Python RIPEMD ===")
print("""
import hashlib
# RIPEMD-160
h = hashlib.new('ripemd160', b"hello").hexdigest()
print(h)
# 108c995b953c8b1c3b1f3b3b3b3b3b3b3b3b3b3b
""")
print("\n=== 8. Security ===")
print("""
MD5:
- Broken
- Do not use
RIPEMD-160:
- Secure
- 80-bit collision
- Acceptable
Recommendation:
- Use SHA-256, SHA-3, BLAKE3
""")
print("\n=== 9. Use Cases ===")
print("""
MD5:
- Checksums (non-security)
- Legacy
RIPEMD-160:
- Bitcoin
- Legacy
Modern:
- SHA-256
- SHA-3
- BLAKE3
""")
print("\n=== 10. Migration ===")
print("""
Migration:
- MD5 β SHA-256
- SHA-1 β SHA-256
- RIPEMD β SHA-256
Example:
Replace MD5
β SHA-256
""")β‘ BLAKE2 & BLAKE3
π― Modern Hash Functions
BLAKE2 and BLAKE3 are modern, fast, and secure hash functions. BLAKE3 is extremely fast, parallelizable, and suitable for all applications.
BLAKE
BLAKE2: Fast. BLAKE3: Faster. Parallel: Yes. Secure: Yes. XOF: BLAKE3. Modern: Recommended.
# BLAKE2 & BLAKE3
print("=== 1. BLAKE2 ===")
print("""
BLAKE2:
- 2015
- Fast
- Secure
- Two variants:
- BLAKE2b (64-bit)
- BLAKE2s (32-bit)
Output:
- 1-64 bytes (BLAKE2b)
- 1-32 bytes (BLAKE2s)
Example:
import hashlib
hashlib.blake2b(b"hello").hexdigest()
""")
print("\n=== 2. BLAKE3 ===")
print("""
BLAKE3:
- 2020
- Very fast
- Parallel
- XOF
Output:
- Variable
- 256-bit default
Example:
import blake3
blake3.blake3(b"hello").hexdigest()
""")
print("\n=== 3. Speed ===")
print("""
Speed:
- MD5: 1x
- SHA-1: 1x
- SHA-256: 0.5x
- SHA-3: 0.3x
- BLAKE2: 3x
- BLAKE3: 10x
BLAKE3 is fastest.
""")
print("\n=== 4. Python BLAKE2 ===")
print("""
import hashlib
# BLAKE2b
h = hashlib.blake2b(b"hello").hexdigest()
print(h)
# BLAKE2s
h = hashlib.blake2s(b"hello").hexdigest()
print(h)
""")
print("\n=== 5. Python BLAKE3 ===")
print("""
# pip install blake3
import blake3
# Hash
h = blake3.blake3(b"hello").hexdigest()
print(h)
# XOF
h = blake3.blake3(b"hello").digest(length=32)
print(h.hex())
""")
print("\n=== 6. Tree Mode ===")
print("""
Tree mode:
- BLAKE3
- Parallel
- Chunks
- Tree
Example:
Input β chunks
β parallel hash
β tree
β root
""")
print("\n=== 7. Keyed Hashing ===")
print("""
Keyed:
- BLAKE2
- BLAKE3
- MAC
- Authentication
Example:
blake2b(b"message", key=b"secret")
""")
print("\n=== 8. Security ===")
print("""
Security:
- BLAKE2: secure
- BLAKE3: secure
- No known attacks
- Recommended
Collision:
- 2^128 (256-bit)
- Secure
""")
print("\n=== 9. Use Cases ===")
print("""
Use cases:
- File integrity
- Password hashing
- HMAC
- Blockchain
- Cryptocurrency
- Content-addressable
- Checksums
""")
print("\n=== 10. Comparison ===")
print(f"{'Algorithm':<15} {'Speed':<15} {'Security':<15} {'Output':<15}")
print("-" * 60)
print(f"{'MD5':<15} {'Fast':<15} {'Broken':<15} {'128':<15}")
print(f"{'SHA-1':<15} {'Fast':<15} {'Broken':<15} {'160':<15}")
print(f"{'SHA-256':<15} {'Medium':<15} {'Secure':<15} {'256':<15}")
print(f"{'SHA-3':<15} {'Medium':<15} {'Secure':<15} {'256':<15}")
print(f"{'BLAKE2':<15} {'Fast':<15} {'Secure':<15} {'256':<15}")
print(f"{'BLAKE3':<15} {'Very fast':<15} {'Secure':<15} {'256':<15}")π³ Merkle Trees
π― Hash Trees
A Merkle tree (hash tree) is a data structure where each leaf node is a hash of a data block, and each non-leaf node is a hash of its children. The root hash (Merkle root) represents all data, enabling efficient and secure verification.
Merkle Trees
Leaves: Data hashes. Internal: Hash of children. Root: Represents all. Proof: Efficient. Bitcoin: Uses. Git: Uses.
# Merkle Trees
print("=== 1. What is a Merkle Tree? ===")
print("""
Merkle:
- Hash tree
- Leaves: data
- Internal: hash
- Root: all
Example:
Root
/ \\
H12 H34
/ \\ / \\
H1 H2 H3 H4
| | | |
D1 D2 D3 D4
""")
print("\n=== 2. Construction ===")
print("""
Construction:
1. Hash each data block
2. Pair hashes
3. Hash pairs
4. Repeat until root
Example:
H1 = SHA256(D1)
H2 = SHA256(D2)
H12 = SHA256(H1 + H2)
Root = SHA256(H12 + H34)
""")
print("\n=== 3. Merkle Proof ===")
print("""
Proof:
- Verify data
- Path to root
- O(log n)
Example:
Verify D1:
H1 = SHA256(D1)
H12 = SHA256(H1 + H2)
Root = SHA256(H12 + H34)
""")
print("\n=== 4. Python Merkle Tree ===")
print("""
import hashlib
def hash_data(data):
\"\"\"Hash data.\"\"\"
if isinstance(data, str):
data = data.encode()
return hashlib.sha256(data).hexdigest()
def build_merkle_tree(leaves):
\"\"\"Build Merkle tree.\"\"\"
if len(leaves) == 1:
return leaves[0]
# Hash leaves
hashes = [hash_data(leaf) for leaf in leaves]
# Build tree
while len(hashes) > 1:
if len(hashes) % 2 == 1:
hashes.append(hashes[-1])
new_hashes = []
for i in range(0, len(hashes), 2):
combined = hashes[i] + hashes[i+1]
new_hashes.append(hash_data(combined))
hashes = new_hashes
return hashes[0]
# Example
# leaves = ["a", "b", "c", "d"]
# root = build_merkle_tree(leaves)
# print(root)
""")
print("\n=== 5. Merkle Proof ===")
print("""
def merkle_proof(leaves, index):
\"\"\"Generate Merkle proof.\"\"\"
hashes = [hash_data(leaf) for leaf in leaves]
proof = []
while len(hashes) > 1:
if len(hashes) % 2 == 1:
hashes.append(hashes[-1])
if index % 2 == 0:
proof.append(hashes[index + 1])
else:
proof.append(hashes[index - 1])
new_hashes = []
for i in range(0, len(hashes), 2):
combined = hashes[i] + hashes[i+1]
new_hashes.append(hash_data(combined))
hashes = new_hashes
index = index // 2
return proof
def verify_proof(leaf, proof, root):
\"\"\"Verify Merkle proof.\"\"\"
h = hash_data(leaf)
for p in proof:
h = hash_data(h + p)
return h == root
# Example
# leaves = ["a", "b", "c", "d"]
# root = build_merkle_tree(leaves)
# proof = merkle_proof(leaves, 0)
# print(verify_proof("a", proof, root))
""")
print("\n=== 6. Bitcoin ===")
print("""
Bitcoin:
- Merkle root
- Block header
- Transactions
- SPV
Example:
Block
β Merkle root
β Header
""")
print("\n=== 7. Ethereum ===")
print("""
Ethereum:
- Merkle Patricia Trie
- State
- Transactions
- Receipts
Example:
State root
β Trie
β Accounts
""")
print("\n=== 8. Git ===")
print("""
Git:
- Merkle tree
- Commits
- Trees
- Blobs
Example:
Commit
β Tree
β Blobs
""")
print("\n=== 9. IPFS ===")
print("""
IPFS:
- Merkle DAG
- Content-addressed
- Files
- Directories
Example:
File
β DAG
β CID
""")
print("\n=== 10. Use Cases ===")
print("""
Use cases:
- Bitcoin
- Ethereum
- Git
- IPFS
- ZFS
- Cassandra
- Certificate transparency
""")π HMAC
π― Hash-Based Message Authentication Code
HMAC is a message authentication code that uses a cryptographic hash function and a secret key. It provides data integrity and authenticity.
HMAC
Key: Secret. Hash: SHA-256. MAC: Authentication. Integrity: Yes. TLS: Uses. JWT: Uses.
# HMAC
print("=== 1. What is HMAC? ===")
print("""
HMAC:
- Hash-based MAC
- Secret key
- Integrity
- Authenticity
Formula:
HMAC(K, m) = H((K β opad) || H((K β ipad) || m))
""")
print("\n=== 2. Properties ===")
print("""
Properties:
- Keyed
- Integrity
- Authenticity
- Resistant to length extension
- Standard
""")
print("\n=== 3. Python HMAC ===")
print("""
import hmac
import hashlib
# HMAC-SHA256
key = b"secret"
message = b"hello"
h = hmac.new(key, message, hashlib.sha256).hexdigest()
print(h)
# Verify
h2 = hmac.new(key, message, hashlib.sha256).hexdigest()
assert h == h2
""")
print("\n=== 4. HMAC in TLS ===")
print("""
TLS:
- HMAC
- Integrity
- Record protocol
- PRF
Example:
TLS record
β HMAC
β Verify
""")
print("\n=== 5. HMAC in JWT ===")
print("""
JWT:
- HMAC-SHA256
- HS256
- Signature
- Token
Example:
Header.Payload
β HMAC
β Signature
""")
print("\n=== 6. API Authentication ===")
print("""
API:
- HMAC
- Request signing
- AWS
- Authentication
Example:
Request
β HMAC
β Signature
""")
print("\n=== 7. Keyed Hash vs HMAC ===")
print("""
Keyed hash:
- H(K || m)
- Length extension
- Not secure
HMAC:
- H((K β opad) || H((K β ipad) || m))
- Secure
- Standard
""")
print("\n=== 8. Python HMAC ===")
print("""
import hmac
import hashlib
def hmac_sha256(key, message):
\"\"\"Compute HMAC-SHA256.\"\"\"
return hmac.new(key, message, hashlib.sha256).hexdigest()
def verify_hmac(key, message, expected):
\"\"\"Verify HMAC.\"\"\"
computed = hmac_sha256(key, message)
return hmac.compare_digest(computed, expected)
# Example
key = b"secret"
message = b"hello"
mac = hmac_sha256(key, message)
print(mac)
print(verify_hmac(key, message, mac))
""")
print("\n=== 9. Use Cases ===")
print("""
Use cases:
- TLS
- JWT
- API authentication
- AWS
- Cookies
- CSRF tokens
""")
print("\n=== 10. Best Practices ===")
print("""
Best practices:
- Use SHA-256
- Random key
- Key length β₯ 32 bytes
- Constant-time compare
- Rotate keys
""")π Password Hashing
π― Secure Password Storage
Password hashing stores passwords securely using slow, salted hash functions like bcrypt, scrypt, and Argon2. Never use MD5 or SHA-256 alone for passwords.
Passwords
Salt: Random. Slow: bcrypt. Memory: Argon2. Cost: Adjustable. Never: MD5. Always: Argon2.
# Password Hashing
print("=== 1. Why Special Hashing? ===")
print("""
Problem:
- MD5/SHA-256 fast
- GPUs: billions/sec
- Brute force
- Rainbow tables
Solution:
- Slow hashing
- Salt
- Memory-hard
""")
print("\n=== 2. Salt ===")
print("""
Salt:
- Random
- Unique per password
- Prevent rainbow tables
- Store with hash
Example:
salt = random(16)
hash = H(salt + password)
""")
print("\n=== 3. bcrypt ===")
print("""
bcrypt:
- 1999
- Blowfish
- Slow
- Cost factor
Example:
import bcrypt
hashed = bcrypt.hashpw(password, bcrypt.gensalt())
""")
print("\n=== 4. scrypt ===")
print("""
scrypt:
- 2009
- Memory-hard
- Slow
- Parallel
Example:
import scrypt
hashed = scrypt.hash(password, salt)
""")
print("\n=== 5. Argon2 ===")
print("""
Argon2:
- 2015
- Password Hashing Competition
- Memory-hard
- Parallel
- Recommended
Variants:
- Argon2d
- Argon2i
- Argon2id (default)
""")
print("\n=== 6. Python bcrypt ===")
print("""
# pip install bcrypt
import bcrypt
# Hash
password = b"secret"
hashed = bcrypt.hashpw(password, bcrypt.gensalt())
print(hashed)
# Verify
assert bcrypt.checkpw(password, hashed)
""")
print("\n=== 7. Python Argon2 ===")
print("""
# pip install argon2-cffi
from argon2 import PasswordHasher
ph = PasswordHasher()
hash = ph.hash("secret")
print(hash)
# Verify
ph.verify(hash, "secret")
""")
print("\n=== 8. Python scrypt ===")
print("""
import hashlib
import secrets
# scrypt
salt = secrets.token_bytes(16)
hashed = hashlib.scrypt(
b"secret",
salt=salt,
n=16384,
r=8,
p=1,
dklen=32
)
print(hashed.hex())
""")
print("\n=== 9. Comparison ===")
print(f"{'Algorithm':<15} {'Year':<10} {'Memory':<15} {'Recommendation':<20}")
print("-" * 60)
print(f"{'MD5':<15} {'1991':<10} {'Low':<15} {'Never':<20}")
print(f"{'SHA-256':<15} {'2001':<10} {'Low':<15} {'Never alone':<20}")
print(f"{'bcrypt':<15} {'1999':<10} {'Low':<15} {'Good':<20}")
print(f"{'scrypt':<15} {'2009':<10} {'Medium':<15} {'Good':<20}")
print(f"{'Argon2':<15} {'2015':<10} {'High':<15} {'Best':<20}")
print("\n=== 10. Best Practices ===")
print("""
Best practices:
- Argon2id
- Salt (16+ bytes)
- Cost (adjust)
- Pepper (optional)
- Never MD5/SHA
- Constant-time compare
""")βοΈ Digital Signatures
π― Signing with Hash Functions
Digital signatures use hash functions to sign messages. The message is hashed, then the hash is signed with a private key. Verification uses the public key.
Signatures
Hash: Message. Sign: Private key. Verify: Public key. ECDSA: Bitcoin. EdDSA: Modern. RSA: Classic.
# Digital Signatures
print("=== 1. What is a Digital Signature? ===")
print("""
Signature:
- Hash message
- Sign hash
- Verify
Properties:
- Authenticity
- Integrity
- Non-repudiation
""")
print("\n=== 2. Signing Process ===")
print("""
Sign:
1. Hash message: h = H(m)
2. Sign hash: s = Sign(h, private_key)
3. Send (m, s)
Verify:
1. Hash message: h = H(m)
2. Verify: Verify(s, h, public_key)
""")
print("\n=== 3. ECDSA ===")
print("""
ECDSA:
- Elliptic Curve
- Bitcoin
- Ethereum
- secp256k1
Example:
from ecdsa import SigningKey, SECP256k1
sk = SigningKey.generate(curve=SECP256k1)
vk = sk.get_verifying_key()
sig = sk.sign(b"message")
assert vk.verify(sig, b"message")
""")
print("\n=== 4. EdDSA ===")
print("""
EdDSA:
- Edwards-curve
- Modern
- Fast
- Ed25519
Example:
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
private = Ed25519PrivateKey.generate()
signature = private.sign(b"message")
""")
print("\n=== 5. RSA ===")
print("""
RSA:
- Classic
- Slow
- Large keys
- PKCS#1
Example:
from Crypto.Signature import pkcs1_15
from Crypto.Hash import SHA256
from Crypto.PublicKey import RSA
""")
print("\n=== 6. Python ECDSA ===")
print("""
# pip install ecdsa
from ecdsa import SigningKey, SECP256k1
# Generate
sk = SigningKey.generate(curve=SECP256k1)
vk = sk.get_verifying_key()
# Sign
message = b"hello"
signature = sk.sign(message)
# Verify
assert vk.verify(signature, message)
""")
print("\n=== 7. Python Ed25519 ===")
print("""
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
# Generate
private = Ed25519PrivateKey.generate()
public = private.public_key()
# Sign
signature = private.sign(b"message")
# Verify
public.verify(signature, b"message")
""")
print("\n=== 8. Bitcoin Signatures ===")
print("""
Bitcoin:
- ECDSA
- secp256k1
- DER encoding
- Low-S
Example:
Transaction
β Hash
β Sign
β Broadcast
""")
print("\n=== 9. Ethereum Signatures ===")
print("""
Ethereum:
- ECDSA
- secp256k1
- EIP-191
- EIP-712
Example:
Message
β Hash
β Sign
β Recover
""")
print("\n=== 10. Use Cases ===")
print("""
Use cases:
- Blockchain
- TLS
- Code signing
- Documents
- Email (S/MIME)
- Software updates
""")βοΈ Blockchain Hashing
π― Hashing in Blockchain
Hash functions are fundamental to blockchain. They link blocks, secure transactions, enable mining, and power addresses and Merkle trees.
Blockchain
Block hash: Links. Mining: PoW. Addresses: Hash. Merkle: Root. Signatures: Hash. SHA-256: Bitcoin.
# Blockchain Hashing
print("=== 1. Block Hash ===")
print("""
Block hash:
- Header hash
- Previous hash
- Links blocks
- Immutable
Example:
SHA256(header)
β Block hash
""")
print("\n=== 2. Mining ===")
print("""
Mining:
- Find nonce
- Hash < target
- PoW
Example:
nonce = 0
while hash > target:
nonce += 1
""")
print("\n=== 3. Bitcoin Addresses ===")
print("""
Bitcoin:
- Public key
- SHA-256
- RIPEMD-160
- Base58Check
Example:
Public key β SHA256
β RIPEMD160
β Address
""")
print("\n=== 4. Ethereum Addresses ===")
print("""
Ethereum:
- Public key
- Keccak-256
- Last 20 bytes
Example:
Public key β Keccak256
β 0x...
""")
print("\n=== 5. Merkle Root ===")
print("""
Merkle:
- Transactions
- Merkle root
- Block header
Example:
Tx1, Tx2, Tx3, Tx4
β Merkle root
""")
print("\n=== 6. Transaction ID ===")
print("""
TXID:
- Transaction hash
- SHA-256 twice
- Bitcoin
Example:
SHA256(SHA256(tx))
β TXID
""")
print("\n=== 7. Python Block ===")
print("""
import hashlib
import json
class Block:
def __init__(self, index, transactions, previous_hash):
self.index = index
self.transactions = transactions
self.previous_hash = previous_hash
self.nonce = 0
self.hash = self.compute_hash()
def compute_hash(self):
block_string = json.dumps({
'index': self.index,
'transactions': self.transactions,
'previous_hash': self.previous_hash,
'nonce': self.nonce
}, sort_keys=True)
return hashlib.sha256(block_string.encode()).hexdigest()
def mine(self, difficulty=4):
target = "0" * difficulty
while not self.hash.startswith(target):
self.nonce += 1
self.hash = self.compute_hash()
# Example
# block = Block(1, ["tx1"], "0")
# block.mine(4)
""")
print("\n=== 8. Python Address ===")
print("""
import hashlib
def bitcoin_address(public_key):
\"\"\"Generate Bitcoin address.\"\"\"
# SHA-256
sha = hashlib.sha256(public_key).digest()
# RIPEMD-160
ripemd = hashlib.new('ripemd160', sha).digest()
# Base58Check
# ...
return ripemd.hex()
def ethereum_address(public_key):
\"\"\"Generate Ethereum address.\"\"\"
# Keccak-256
from Crypto.Hash import keccak
k = keccak.new(digest_bits=256)
k.update(public_key)
return '0x' + k.hexdigest()[-40:]
# Example
# print(bitcoin_address(b"public_key"))
# print(ethereum_address(b"public_key"))
""")
print("\n=== 9. Use Cases ===")
print("""
Use cases:
- Bitcoin
- Ethereum
- Blockchain
- Cryptocurrency
- Smart contracts
- DeFi
""")
print("\n=== 10. Best Practices ===")
print("""
Best practices:
- SHA-256 (Bitcoin)
- Keccak-256 (Ethereum)
- Double SHA-256
- Merkle root
- Signatures
""")π Python Implementation
π― Hashing in Python
Python provides built-in support for hash functions through the hashlib module, and external libraries for advanced use cases.
Python
hashlib: Built-in. hmac: Built-in. bcrypt: Passwords. argon2: Passwords. blake3: Fast. ecdsa: Signatures.
# Python Implementation
print("=== 1. hashlib ===")
print("""
import hashlib
# SHA-256
h = hashlib.sha256(b"hello").hexdigest()
print(h)
# SHA-3
h = hashlib.sha3_256(b"hello").hexdigest()
print(h)
# BLAKE2
h = hashlib.blake2b(b"hello").hexdigest()
print(h)
# MD5
h = hashlib.md5(b"hello").hexdigest()
print(h)
""")
print("\n=== 2. Incremental ===")
print("""
import hashlib
h = hashlib.sha256()
h.update(b"hello")
h.update(b" world")
print(h.hexdigest())
""")
print("\n=== 3. HMAC ===")
print("""
import hmac
import hashlib
mac = hmac.new(b"key", b"message", hashlib.sha256).hexdigest()
print(mac)
""")
print("\n=== 4. bcrypt ===")
print("""
# pip install bcrypt
import bcrypt
password = b"secret"
hashed = bcrypt.hashpw(password, bcrypt.gensalt())
print(hashed)
assert bcrypt.checkpw(password, hashed)
""")
print("\n=== 5. Argon2 ===")
print("""
# pip install argon2-cffi
from argon2 import PasswordHasher
ph = PasswordHasher()
hash = ph.hash("secret")
print(hash)
ph.verify(hash, "secret")
""")
print("\n=== 6. BLAKE3 ===")
print("""
# pip install blake3
import blake3
h = blake3.blake3(b"hello").hexdigest()
print(h)
""")
print("\n=== 7. ECDSA ===")
print("""
# pip install ecdsa
from ecdsa import SigningKey, SECP256k1
sk = SigningKey.generate(curve=SECP256k1)
vk = sk.get_verifying_key()
signature = sk.sign(b"message")
assert vk.verify(signature, b"message")
""")
print("\n=== 8. Merkle Tree ===")
print("""
import hashlib
def hash_data(data):
if isinstance(data, str):
data = data.encode()
return hashlib.sha256(data).hexdigest()
def build_merkle_tree(leaves):
hashes = [hash_data(leaf) for leaf in leaves]
while len(hashes) > 1:
if len(hashes) % 2 == 1:
hashes.append(hashes[-1])
new_hashes = []
for i in range(0, len(hashes), 2):
new_hashes.append(hash_data(hashes[i] + hashes[i+1]))
hashes = new_hashes
return hashes[0]
# Example
# root = build_merkle_tree(["a", "b", "c", "d"])
# print(root)
""")
print("\n=== 9. Complete Example ===")
print("""
import hashlib
import hmac
import bcrypt
import blake3
from ecdsa import SigningKey, SECP256k1
def demo():
\"\"\"Demonstrate hash functions.\"\"\"
# SHA-256
print(f"SHA-256: {hashlib.sha256(b'hello').hexdigest()}")
# SHA-3
print(f"SHA-3: {hashlib.sha3_256(b'hello').hexdigest()}")
# BLAKE2
print(f"BLAKE2: {hashlib.blake2b(b'hello').hexdigest()}")
# BLAKE3
print(f"BLAKE3: {blake3.blake3(b'hello').hexdigest()}")
# HMAC
print(f"HMAC: {hmac.new(b'key', b'msg', hashlib.sha256).hexdigest()}")
# bcrypt
print(f"bcrypt: {bcrypt.hashpw(b'pass', bcrypt.gensalt())}")
# ECDSA
sk = SigningKey.generate(curve=SECP256k1)
vk = sk.get_verifying_key()
sig = sk.sign(b"msg")
print(f"Verify: {vk.verify(sig, b'msg')}")
# demo()
""")
print("\n=== 10. Libraries ===")
print("""
Libraries:
- hashlib (built-in)
- hmac (built-in)
- bcrypt
- argon2-cffi
- blake3
- ecdsa
- pycryptodome
- cryptography
- passlib
""")βοΈ Comparison
π― Which Hash Function?
Choosing the right hash function depends on your use case, security requirements, and performance needs.
Comparison
SHA-256: Standard. SHA-3: Modern. BLAKE3: Fastest. Argon2: Passwords. MD5: Broken. bcrypt: Passwords.
| Algorithm | Output | Speed | Security | Best For |
|---|---|---|---|---|
| MD5 | 128-bit | Fast | Broken | Checksums only |
| SHA-1 | 160-bit | Fast | Broken | Legacy only |
| SHA-256 | 256-bit | Medium | Secure | General, Bitcoin |
| SHA-512 | 512-bit | Medium | Secure | General |
| SHA-3 | 256-bit | Medium | Secure | Ethereum, modern |
| BLAKE2 | 256-bit | Fast | Secure | Fast hashing |
| BLAKE3 | 256-bit | Very fast | Secure | Fastest |
| RIPEMD-160 | 160-bit | Fast | Secure | Bitcoin |
| bcrypt | 192-bit | Slow | Secure | Passwords |
| Argon2 | Variable | Slow | Secure | Passwords (best) |
# Decision Guide
print("=== Choose Hash by Use Case ===")
print("""
General purpose:
β SHA-256
β SHA-3
β BLAKE3
Blockchain (Bitcoin):
β SHA-256
β RIPEMD-160
Blockchain (Ethereum):
β Keccak-256 (SHA-3)
Passwords:
β Argon2id (best)
β bcrypt
β scrypt
HMAC:
β HMAC-SHA256
Fast hashing:
β BLAKE3
β BLAKE2
Checksums (non-security):
β MD5
β CRC32
Merkle trees:
β SHA-256
β BLAKE3
""")
print("\n=== Speed Comparison ===")
print(f"{'Algorithm':<20} {'Speed (MB/s)':<20} {'Relative':<15}")
print("-" * 55)
print(f"{'MD5':<20} {'500':<20} {'1x':<15}")
print(f"{'SHA-1':<20} {'500':<20} {'1x':<15}")
print(f"{'SHA-256':<20} {'250':<20} {'0.5x':<15}")
print(f"{'SHA-512':<20} {'500':<20} {'1x':<15}")
print(f"{'SHA-3':<20} {'150':<20} {'0.3x':<15}")
print(f"{'BLAKE2':<20} {'1000':<20} {'2x':<15}")
print(f"{'BLAKE3':<20} {'5000':<20} {'10x':<15}")
print("\n=== Security Comparison ===")
print(f"{'Algorithm':<20} {'Preimage':<20} {'Collision':<20}")
print("-" * 60)
print(f"{'MD5':<20} {'2^128':<20} {'2^64 (broken)':<20}")
print(f"{'SHA-1':<20} {'2^160':<20} {'2^80 (broken)':<20}")
print(f"{'SHA-256':<20} {'2^256':<20} {'2^128':<20}")
print(f"{'SHA-512':<20} {'2^512':<20} {'2^256':<20}")
print(f"{'SHA-3':<20} {'2^256':<20} {'2^128':<20}")
print(f"{'BLAKE3':<20} {'2^256':<20} {'2^128':<20}")
print("\n=== When to Use Which ===")
print("""
USE SHA-256:
β General purpose
β Bitcoin
β HMAC
β Merkle trees
USE SHA-3:
β Modern
β Ethereum
β Post-quantum
β Backup
USE BLAKE3:
β Speed
β Parallel
β General
USE Argon2:
β Passwords
β Best security
USE bcrypt:
β Passwords
β Legacy
USE MD5:
β Checksums only
β Never for security
""")β FAQ
π― Frequently Asked Questions
Answers to common questions about cryptographic hash functions.
What is a cryptographic hash function?
A cryptographic hash function is a mathematical algorithm that takes an input (or "message") and produces a fixed-size output (hash). It is deterministic, one-way (preimage resistant), collision-resistant, and exhibits the avalanche effect. Common examples include SHA-256, SHA-3, and BLAKE3.
What is the difference between SHA-256 and SHA-3?
SHA-256 belongs to the SHA-2 family and uses the MerkleβDamgΓ₯rd construction. SHA-3 (Keccak) uses a sponge construction, providing different security properties and resistance to length extension attacks. Both produce 256-bit hashes and are considered secure.
Why is MD5 considered insecure?
MD5 is considered insecure because practical collision attacks have been demonstrated. Two different inputs can produce the same MD5 hash, making it unsuitable for security-critical applications like digital signatures. Use SHA-256, SHA-3, or BLAKE3 instead.
What is a Merkle tree?
A Merkle tree (hash tree) is a data structure where each leaf node is a hash of a data block, and each non-leaf node is a hash of its children. The root hash (Merkle root) represents all data, enabling efficient and secure verification of large datasets. It is used in Bitcoin, Ethereum, and Git.
What is the best password hashing algorithm?
The best password hashing algorithms are Argon2 (winner of the Password Hashing Competition), bcrypt, and scrypt. They are designed to be slow and memory-intensive, resisting brute-force and GPU attacks. Never use MD5 or SHA-256 alone for passwords.
What is HMAC?
HMAC (Hash-Based Message Authentication Code) is a message authentication code that uses a cryptographic hash function and a secret key. It provides data integrity and authenticity. HMAC-SHA256 is commonly used in TLS, JWT, and API authentication.
What is the avalanche effect?
The avalanche effect means that a small change in the input (e.g., flipping one bit) produces a significantly different hash output (approximately 50% of bits change). This property ensures that similar inputs do not produce similar hashes.
What is a salt in password hashing?
A salt is a random value added to a password before hashing. It ensures that identical passwords produce different hashes, preventing rainbow table attacks. Salts are stored alongside the hash and do not need to be secret.
What is a collision attack?
A collision attack attempts to find two different inputs that produce the same hash output. Due to the birthday paradox, collision resistance is about 2^(n/2) for an n-bit hash. MD5 (128-bit) and SHA-1 (160-bit) are broken; SHA-256 (256-bit) remains secure.
What is length extension attack?
A length extension attack exploits the MerkleβDamgΓ₯rd construction (used in MD5, SHA-1, SHA-2) to compute H(message || padding || extension) without knowing the message. SHA-3 and BLAKE3 are resistant due to their sponge/tree constructions.
# Quick Reference
print("=== Quick Answers ===")
print("""
Q: What is a hash function?
A: One-way fixed-size output.
Q: Best general hash?
A: SHA-256, SHA-3, BLAKE3.
Q: Fastest hash?
A: BLAKE3.
Q: Broken hashes?
A: MD5, SHA-1.
Q: Best password hash?
A: Argon2id.
Q: What is HMAC?
A: Keyed hash for authentication.
Q: What is Merkle tree?
A: Hash tree for verification.
Q: What is salt?
A: Random value added to password.
Q: What is avalanche?
A: Small change β large difference.
Q: Bitcoin hash?
A: SHA-256.
Q: Ethereum hash?
A: Keccak-256.
Q: Free?
A: Yes β all algorithms are open.
""")π Future
π― Where Hashing Is Going
The future is post-quantum, faster, parallel, memory-hard, and AI-optimized.
Future Directions
Post-quantum: Quantum-resistant. Faster: BLAKE3. Parallel: Multi-core. Memory: Argon2. AI: Optimization. Hardware: ASICs.
# Future of Hashing
print("=== 1. Post-Quantum ===")
print("""
Trend: Post-quantum
Current:
- SHA-256
- SHA-3
- BLAKE3
Future:
- Quantum-resistant
- Larger outputs
- New designs
Examples:
- SPHINCS+
- XMSS
- LMS
Future:
- Quantum-safe
- Larger hashes
- New standards
""")
print("\n=== 2. Faster Hashing ===")
print("""
Trend: Speed
Current:
- BLAKE3: 5 GB/s
Future:
- 10+ GB/s
- Parallel
- Hardware
Examples:
- BLAKE3
- xxHash
- HighwayHash
Future:
- Real-time
- Streaming
- Parallel
""")
print("\n=== 3. Memory-Hard ===")
print("""
Trend: Memory-hard
Current:
- Argon2
- scrypt
Future:
- More memory
- Better resistance
- GPU-resistant
Examples:
- Argon2id
- Balloon
- Catena
Future:
- Passwords
- Resistance
- Standard
""")
print("\n=== 4. AI Optimization ===")
print("""
Trend: AI + hashing
Ideas:
- AI-optimized
- Neural hashing
- Learned functions
Examples:
- Deep learning
- Neural networks
- Optimization
Future:
- AI-designed
- Faster
- Better
""")
print("\n=== 5. Hardware ===")
print("""
Trend: Hardware
Current:
- CPU
- GPU
Future:
- ASICs
- FPGA
- Quantum
Examples:
- Bitcoin ASICs
- SHA-256
- BLAKE3
Future:
- Specialized
- Fast
- Efficient
""")
print("\n=== 6. Standards ===")
print("""
Trend: Standards
Current:
- NIST
- SHA-3
Future:
- Post-quantum
- New standards
- International
Examples:
- NIST PQC
- ISO
- IETF
Future:
- Updated
- Quantum-safe
- Global
""")
# TIMELINE
print("\n=== Timeline ===")
print("""
Past:
1989: MD2
1991: MD5
1993: SHA-0
1995: SHA-1
2001: SHA-2
2012: SHA-3
2015: BLAKE2
2020: BLAKE3
Present:
2025: BLAKE3
2026: Post-quantum
Future:
2030: Quantum-safe
2035: AI-optimized
2040: ?
""")
print("\n=== Final Thoughts ===")
print("""
Hash functions are:
- Fundamental
- Everywhere
- Evolving
- Critical
The future is:
- Post-quantum
- Faster
- Parallel
- Memory-hard
- AI-optimized
- Hardware-accelerated
"The hash is the fingerprint of data."
Keep exploring. Keep building.
""")