๐ง Haptics Fundamentals
๐ฏ The Science of Touch
Haptics is the science and technology of tactile and kinesthetic communication โ simulating the sense of touch. It encompasses tactile feedback (vibration, texture, pressure) and kinesthetic feedback (force, position, movement). Haptics is essential for immersive AR/VR, teleoperation, surgical robotics, and accessibility.
Why Haptics Matters
Presence: Touch completes the immersive experience. Information: Conveys texture, weight, temperature, stiffness. Feedback: Confirms actions (button press, collision). Accessibility: Enables communication for deafblind users. Training: Surgical simulation, skill acquisition. Teleoperation: Feel what a robot feels.
๐ Tactile๐ Kinesthetic๐ Force๐ Vibration
# Haptic feedback pipeline (conceptual)
class HapticSystem:
def __init__(self):
self.actuators = [] # vibration motors, force feedback, etc.
self.sensors = [] # position, force, touch sensors
self.renderer = HapticRenderer()
def update(self, dt):
# 1. Read sensors (user input, collision)
sensor_data = self.read_sensors()
# 2. Compute haptic feedback
feedback = self.renderer.compute(sensor_data)
# 3. Send to actuators
for actuator, signal in zip(self.actuators, feedback):
actuator.set(signal)
def render_contact(self, object_properties, interaction):
"""Render haptic feedback for contact with virtual object."""
stiffness = object_properties['stiffness']
friction = object_properties['friction']
penetration = interaction['penetration_depth']
# Spring-damper model
force = stiffness * penetration - friction * interaction['velocity']
return force๐๏ธ Human Touch Biology
๐ฏ How we sense touch
The human hand contains ~17,000 mechanoreceptors. Key types: Merkel cells (pressure), Meissner corpuscles (flutter), Pacinian corpuscles (vibration), Ruffini endings (stretch). Spatial acuity varies โ fingertips are most sensitive.
Mechanoreceptors
Pacinian: 20-1000 Hz, detects vibration. Meissner: 5-50 Hz, detects flutter/grip. Merkel: 0.5-15 Hz, detects pressure/texture. Ruffini: 0.5-15 Hz, detects stretch/skin deformation. Nociceptors: Pain. Thermoreceptors: Temperature. Two-point discrimination: fingertips ~2mm, palm ~10mm, back ~40mm.
๐ณ Pacinian๐ Meissner๐ Merkelโ๏ธ Ruffini
# Mechanoreceptor models for haptic rendering
import numpy as np
class MechanoreceptorModel:
def __init__(self):
# Adaptation rates and frequency sensitivities
self.receptors = {
'pacinian': {'adaptation': 'fast', 'freq_range': (20, 1000), 'threshold': 0.001},
'meissner': {'adaptation': 'fast', 'freq_range': (5, 50), 'threshold': 0.01},
'merkel': {'adaptation': 'slow', 'freq_range': (0.5, 15), 'threshold': 0.02},
'ruffini': {'adaptation': 'slow', 'freq_range': (0.5, 15), 'threshold': 0.05}
}
def compute_activation(self, stimulus_freq, stimulus_amp):
"""Compute receptor activation for a given stimulus."""
activations = {}
for name, props in self.receptors.items():
f_min, f_max = props['freq_range']
if f_min <= stimulus_freq <= f_max:
# Gaussian tuning curve
center = (f_min + f_max) / 2
bandwidth = (f_max - f_min) / 2
tuning = np.exp(-((stimulus_freq - center) ** 2) / (2 * bandwidth ** 2))
# Threshold
if stimulus_amp > props['threshold']:
activations[name] = tuning * (stimulus_amp - props['threshold'])
else:
activations[name] = 0
else:
activations[name] = 0
return activations
# Two-point discrimination test
def two_point_threshold(body_part):
thresholds = {
'fingertip': 2.0, # mm
'palm': 10.0,
'forearm': 35.0,
'back': 40.0,
'calf': 45.0
}
return thresholds.get(body_part, 40.0)โ๏ธ Actuators & Technologies
๐ฏ Creating touch sensations
Haptic actuators: ERM (Eccentric Rotating Mass), LRA (Linear Resonant Actuator), piezo, voice coils, electrostatic, ultrasonic, and pneumatic.
Actuator Comparison
ERM: Cheap, high amplitude, slow response. LRA: Fast response, crisp, low power (phones). Piezo: Very fast, high frequency, low amplitude. Voice coil: Wide frequency range, high fidelity. Electrostatic: Thin, texture rendering. Ultrasonic: Mid-air, no contact. Pneumatic: Force feedback, bulky.
๐ ERMโก LRA๐ Piezo๐ Ultrasonic
// Haptic actuator control (Arduino)
#include
// LRA control with DRV2605L
#define DRV2605_ADDR 0x5A
void setup() {
Wire.begin();
// Initialize DRV2605
Wire.beginTransmission(DRV2605_ADDR);
Wire.write(0x01); // Mode register
Wire.write(0x00); // Internal trigger
Wire.endTransmission();
// Set to LRA mode
Wire.beginTransmission(DRV2605_ADDR);
Wire.write(0x1A); // Feedback control
Wire.write(0xB6); // LRA mode, closed loop
Wire.endTransmission();
}
void playEffect(uint8_t effect) {
// Set waveform
Wire.beginTransmission(DRV2605_ADDR);
Wire.write(0x04); // Waveform sequence
Wire.write(effect);
Wire.endTransmission();
// Trigger
Wire.beginTransmission(DRV2605_ADDR);
Wire.write(0x0C); // GO register
Wire.write(0x01);
Wire.endTransmission();
}
void loop() {
playEffect(1); // Strong click
delay(500);
playEffect(47); // Buzz
delay(500);
}
// Piezo haptic control (higher frequency)
void playPiezo(float frequency, float amplitude, int duration_ms) {
unsigned long start = millis();
while (millis() - start < duration_ms) {
int period = 1000000 / frequency;
digitalWrite(PIEZO_PIN, HIGH);
delayMicroseconds(period / 2);
digitalWrite(PIEZO_PIN, LOW);
delayMicroseconds(period / 2);
}
} ๐ณ Vibration Haptics
๐ฏ The most common haptic feedback
Vibration haptics use ERM or LRA motors to create tactile sensations. It's ubiquitous in phones, controllers, and wearables. Techniques: amplitude modulation, frequency modulation, waveform design.
Vibration Design
Amplitude: Strength of vibration. Frequency: Speed of vibration (Hz). Duration: Length of vibration. Envelope: Attack, sustain, decay. Waveforms: Click, tick, buzz, pulse. HD Haptics: High-definition vibrations with rich textures (iPhone Taptic Engine, DualSense).
๐ Amplitude๐ต Frequencyโฑ๏ธ Duration๐จ Waveform
// Vibration patterns (Android)
import android.os.Vibrator
import android.os.VibrationEffect
class HapticManager(private val vibrator: Vibrator) {
fun playClick() {
if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.O) {
val effect = VibrationEffect.createOneShot(10, VibrationEffect.DEFAULT_AMPLITUDE)
vibrator.vibrate(effect)
} else {
vibrator.vibrate(10)
}
}
fun playPattern() {
// Pattern: wait, vibrate, wait, vibrate
val timings = longArrayOf(0, 100, 50, 100, 50, 200)
val amplitudes = intArrayOf(0, 255, 0, 128, 0, 255)
if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.O) {
val effect = VibrationEffect.createWaveform(timings, amplitudes, -1)
vibrator.vibrate(effect)
}
}
fun playHeartbeat() {
// Heartbeat pattern: two pulses
val timings = longArrayOf(0, 50, 100, 50, 500)
val amplitudes = intArrayOf(0, 200, 0, 150, 0)
val effect = VibrationEffect.createWaveform(timings, amplitudes, 0) // repeat
vibrator.vibrate(effect)
}
}
// iOS Core Haptics
import CoreHaptics
class HapticEngine {
var engine: CHHapticEngine?
func setup() {
engine = try? CHHapticEngine()
try? engine?.start()
}
func playTap() {
let intensity = CHHapticEventParameter(parameterID: .hapticIntensity, value: 1.0)
let sharpness = CHHapticEventParameter(parameterID: .hapticSharpness, value: 0.5)
let event = CHHapticEvent(eventType: .hapticTransient,
parameters: [intensity, sharpness],
relativeTime: 0)
let pattern = try? CHHapticPattern(events: [event], parameters: [])
let player = try? engine?.makePlayer(with: pattern!)
try? player?.start(atTime: 0)
}
}๐ฆพ Force Feedback
๐ฏ Resisting and assisting motion
Force feedback applies forces to the user's hand/body, simulating weight, stiffness, and resistance. Used in surgical robots, flight simulators, and haptic gloves.
Force Feedback Types
Impedance: Render force based on position (spring-damper). Admittance: Render motion based on force. Degrees of freedom: 1-DOF (steering wheel) to 6-DOF (robotic arm). Stiffness: How hard the virtual object feels. Damping: Resistance to motion. Inertia: Simulated mass.
๐ฆพ Impedance๐ Admittance๐ฉ Stiffness๐๏ธ Inertia
// Force feedback with spring-damper model
class ForceFeedback {
constructor() {
this.stiffness = 500; // N/m
this.damping = 10; // Ns/m
this.virtualObjectPosition = { x: 0, y: 0, z: 0 };
this.virtualObjectRadius = 0.05; // 5cm
}
computeForce(handlePosition, handleVelocity) {
// Distance from handle to object center
const dx = handlePosition.x - this.virtualObjectPosition.x;
const dy = handlePosition.y - this.virtualObjectPosition.y;
const dz = handlePosition.z - this.virtualObjectPosition.z;
const distance = Math.sqrt(dx*dx + dy*dy + dz*dz);
// Check if inside object
const penetration = this.virtualObjectRadius - distance;
if (penetration > 0) {
// Normal direction (from object center to handle)
const nx = dx / distance;
const ny = dy / distance;
const nz = dz / distance;
// Spring force
const springForce = this.stiffness * penetration;
// Damping force (opposes velocity)
const dampingForce = -this.damping * (
handleVelocity.x * nx +
handleVelocity.y * ny +
handleVelocity.z * nz
);
// Total force
const totalForce = springForce + dampingForce;
return {
x: totalForce * nx,
y: totalForce * ny,
z: totalForce * nz
};
}
return { x: 0, y: 0, z: 0 };
}
}
// Haptic rendering loop at 1kHz
const forceFeedback = new ForceFeedback();
const hapticLoop = setInterval(() => {
const handlePos = getHandlePosition();
const handleVel = getHandleVelocity();
const force = forceFeedback.computeForce(handlePos, handleVel);
setOutputForce(force);
}, 1); // 1ms = 1kHz๐ Ultrasonic Mid-Air Haptics
๐ฏ Touching without touching
Ultrasonic mid-air haptics uses phased arrays of ultrasound transducers to create pressure points in the air. Users feel virtual objects without wearing anything. Used in automotive, public displays, and AR/VR.
How It Works
Phased array: 100+ ultrasonic transducers. Focal point: Constructive interference creates pressure. Modulation: Amplitude, frequency, spatiotemporal. Sensations: Point, line, circle, texture, vibration. Range: 10-30cm above array. Companies: Ultraleap (formerly Ultrahaptics).
๐ Phased Array๐ฏ Focal Point๐ซ๏ธ Mid-Airโ No Contact
# Ultrasonic phased array simulation
import numpy as np
class UltrasonicArray:
def __init__(self, num_transducers=256, frequency=40000):
self.num_transducers = num_transducers
self.frequency = frequency # 40 kHz
self.wavelength = 343.0 / frequency # speed of sound / frequency
self.transducers = self._arrange_transducers()
def _arrange_transducers(self):
"""Arrange transducers in a circular pattern."""
transducers = []
for i in range(self.num_transducers):
angle = 2 * np.pi * i / self.num_transducers
radius = 0.1 # 10cm array radius
x = radius * np.cos(angle)
y = radius * np.sin(angle)
transducers.append((x, y, 0))
return np.array(transducers)
def compute_phases(self, focal_point):
"""Compute phase delays for each transducer to focus at focal_point."""
focal_point = np.array(focal_point)
phases = []
for transducer in self.transducers:
# Distance from transducer to focal point
distance = np.linalg.norm(focal_point - transducer)
# Phase delay = 2ฯ * distance / wavelength
phase = (2 * np.pi * distance / self.wavelength) % (2 * np.pi)
phases.append(phase)
return np.array(phases)
def compute_pressure_field(self, focal_point, resolution=50):
"""Compute pressure field in a plane above the array."""
phases = self.compute_phases(focal_point)
# Sample points in a plane at z = focal_point[2]
x = np.linspace(-0.1, 0.1, resolution)
y = np.linspace(-0.1, 0.1, resolution)
X, Y = np.meshgrid(x, y)
Z = np.full_like(X, focal_point[2])
pressure = np.zeros_like(X, dtype=complex)
for i, transducer in enumerate(self.transducers):
# Distance from transducer to each point
R = np.sqrt((X - transducer[0])**2 + (Y - transducer[1])**2 + (Z - transducer[2])**2)
# Pressure contribution (simplified)
amplitude = 1.0 / R # spherical spreading
phase = 2 * np.pi * R / self.wavelength + phases[i]
pressure += amplitude * np.exp(1j * phase)
return np.abs(pressure)
# Usage
array = UltrasonicArray(num_transducers=256)
focal_point = (0, 0, 0.15) # 15cm above center
pressure_field = array.compute_pressure_field(focal_point)
print(f"Max pressure: {pressure_field.max():.2f}")
print(f"Focal point pressure: {pressure_field[25, 25]:.2f}")๐ก๏ธ Thermal Haptics
๐ฏ Feeling hot and cold
Thermal haptics simulates temperature sensations using Peltier elements (thermoelectric coolers). It adds realism to virtual experiences โ feeling a cold drink, warm sun, or hot surface.
Thermal Feedback
Peltier: Heat pump โ one side hot, other cold. Range: 0-50ยฐC safe range. Response time: 1-5 seconds (slow). Applications: Virtual drinks, weather, material identification. Challenges: Slow, power-hungry, safety. Combined: Thermal + vibration for richer sensations.
๐ฅ Hotโ๏ธ Coldโก Peltier๐ข Slow
// Thermal haptic control (Peltier + Arduino)
#include
#define PELTIER_PIN 9
#define TEMP_SENSOR_PIN A0
double setpoint = 25.0; // target temperature ยฐC
double input, output;
double Kp = 10, Ki = 0.5, Kd = 1;
PID thermalPID(&input, &output, &setpoint, Kp, Ki, Kd, DIRECT);
void setup() {
Serial.begin(9600);
thermalPID.SetMode(AUTOMATIC);
thermalPID.SetOutputLimits(-255, 255); // bidirectional
thermalPID.SetSampleTime(100); // 10Hz
}
void loop() {
// Read temperature
int sensorValue = analogRead(TEMP_SENSOR_PIN);
input = (sensorValue / 1024.0) * 500.0; // LM35: 10mV/ยฐC
// Compute PID
thermalPID.Compute();
// Drive Peltier (H-bridge for bidirectional)
if (output > 0) {
analogWrite(PELTIER_PIN, (int)output); // heating
} else {
analogWrite(PELTIER_PIN, (int)-output); // cooling
}
Serial.print("Temp: ");
Serial.print(input);
Serial.print("ยฐC, Output: ");
Serial.println(output);
delay(100);
}
// Thermal pattern: simulate cold drink
void simulateColdDrink() {
setpoint = 5.0; // cold
delay(3000);
setpoint = 25.0; // back to neutral
}
// Thermal pattern: simulate hot coffee
void simulateHotCoffee() {
setpoint = 45.0; // hot
delay(2000);
setpoint = 25.0; // back to neutral
} โก Electrotactile Stimulation
๐ฏ Direct nerve stimulation
Electrotactile stimulation uses small electrical currents to stimulate touch receptors directly, creating sensations without mechanical actuators. Used in sensory substitution and haptic feedback.
Electrical Stimulation
TENS: Transcutaneous electrical nerve stimulation. Electrodes: On skin, stimulate afferent nerves. Sensations: Tingling, vibration, pressure, pain (if too strong). Frequency: 1-100 Hz. Applications: Sensory substitution (vision-to-touch), haptic feedback, pain relief. Safety: Current-limited, medical-grade.
โก TENS๐ง Nerve๐ Electrode๐ Frequency
// Electrotactile stimulation (safety-critical!)
class ElectrotactileStimulator {
constructor() {
this.maxCurrent = 20; // mA (safety limit)
this.maxFrequency = 100; // Hz
this.pulseWidth = 200; // microseconds
this.electrodes = [];
}
addElectrode(position) {
this.electrodes.push({
position: position,
active: false,
current: 0
});
}
stimulate(electrodeIndex, intensity, frequency) {
// Safety checks
if (intensity > this.maxCurrent) {
throw new Error("Current exceeds safety limit");
}
if (frequency > this.maxFrequency) {
throw new Error("Frequency exceeds safety limit");
}
const electrode = this.electrodes[electrodeIndex];
electrode.active = true;
electrode.current = intensity;
// Generate biphasic pulses (balanced charge)
this.generatePulses(electrode, frequency);
}
generatePulses(electrode, frequency) {
const period = 1000000 / frequency; // microseconds
const interval = setInterval(() => {
if (!electrode.active) {
clearInterval(interval);
return;
}
// Biphasic pulse: positive then negative
this.applyCurrent(electrode, electrode.current, this.pulseWidth);
setTimeout(() => {
this.applyCurrent(electrode, -electrode.current, this.pulseWidth);
}, this.pulseWidth);
}, period);
}
stop(electrodeIndex) {
this.electrodes[electrodeIndex].active = false;
this.electrodes[electrodeIndex].current = 0;
}
}
// Sensory substitution: vision to touch
function visionToTouch(imageData, electrodeArray) {
// Downsample image to electrode grid
const gridSize = Math.sqrt(electrodeArray.length);
const cellWidth = imageData.width / gridSize;
const cellHeight = imageData.height / gridSize;
for (let i = 0; i < electrodeArray.length; i++) {
const row = Math.floor(i / gridSize);
const col = i % gridSize;
// Average brightness in cell
let sum = 0, count = 0;
for (let y = row * cellHeight; y < (row + 1) * cellHeight; y++) {
for (let x = col * cellWidth; x < (col + 1) * cellWidth; x++) {
const idx = (y * imageData.width + x) * 4;
sum += (imageData.data[idx] + imageData.data[idx + 1] + imageData.data[idx + 2]) / 3;
count++;
}
}
const brightness = sum / count;
// Map brightness to stimulation intensity
const intensity = (brightness / 255) * 15; // max 15mA
if (intensity > 1) {
electrodeArray[i].stimulate(intensity, 50);
} else {
electrodeArray[i].stop();
}
}
}๐งค Haptic Gloves & Exoskeletons
๐ฏ Wearing the virtual world
Haptic gloves and exoskeletons provide force feedback and tactile sensations to the hand and fingers. Used in VR training, teleoperation, and medical simulation.
Glove Technologies
Vibration gloves: Buzz on contact. Force feedback gloves: Resist finger movement (exoskeleton). Pneumatic gloves: Air bladders for pressure. Electrotactile gloves: Electrical stimulation. Thermal gloves: Hot/cold. Companies: HaptX, Dexmo, SenseGlove, Manus.
๐งค Vibration๐ฆพ Exoskeleton๐จ Pneumaticโก Electro
# Haptic glove control (Unity + SenseGlove)
using UnityEngine;
using SenseGloveCs;
public class HapticGloveController : MonoBehaviour {
public SenseGlove glove;
public float forceMultiplier = 1.0f;
void Start() {
// Connect to glove
SenseGloveCs.SenseGlove.Connect();
}
void Update() {
// Get finger positions
for (int i = 0; i < 5; i++) {
float flexion = glove.GetFingerFlexion(i);
float abduction = glove.GetFingerAbduction(i);
// Apply to virtual hand
virtualHand.SetFinger(i, flexion, abduction);
}
// Check for collisions
CheckCollisions();
}
void CheckCollisions() {
// For each fingertip, check collision with virtual object
for (int i = 0; i < 5; i++) {
Vector3 fingertipPos = virtualHand.GetFingertipPosition(i);
if (Physics.CheckSphere(fingertipPos, 0.01f)) {
// Collision detected โ compute force
float penetration = ComputePenetration(fingertipPos);
float force = penetration * 10.0f * forceMultiplier;
// Send force feedback to glove
glove.SetForceFeedback(i, force);
// Play haptic effect
glove.PlayHapticEffect(i, HapticEffect.Contact);
}
}
}
void OnDestroy() {
SenseGloveCs.SenseGlove.Disconnect();
}
}
// Exoskeleton force feedback (Dexmo)
public class ExoskeletonController : MonoBehaviour {
public DexmoExoskeleton exoskeleton;
void ApplyForce(int fingerIndex, float force) {
// Force range: 0-5N
force = Mathf.Clamp(force, 0, 5);
exoskeleton.SetBrakeForce(fingerIndex, force);
}
void OnCollisionEnter(Collision collision) {
// Determine which finger hit
int finger = GetFingerFromCollider(collision.collider);
// Compute force based on object stiffness
float objectStiffness = collision.gameObject.GetComponent().stiffness;
float penetration = collision.contacts[0].separation;
float force = objectStiffness * Mathf.Abs(penetration);
ApplyForce(finger, force);
}
} ๐จ Haptic Rendering
๐ฏ Computing touch sensations
Haptic rendering computes forces and vibrations based on virtual object properties and user interaction. It requires high update rates (1kHz) for stable force feedback.
Haptic Rendering Pipeline
1. Collision detection: Find contacts (spheres, meshes). 2. Force computation: Spring-damper, penalty, constraint-based. 3. Force filtering: Smooth, stabilize. 4. Actuator control: Convert force to actuator signals. 5. Update rate: 1kHz for force, 100Hz for vibration. Stability: Critical โ oscillations if too stiff.
๐ฏ Collision๐ฉ Forceโก 1kHz๐ Stability
# Haptic rendering with god-object method
import numpy as np
class HapticRenderer:
def __init__(self):
self.stiffness = 1000.0 # N/m
self.damping = 5.0 # Ns/m
self.max_force = 10.0 # N (safety)
def render(self, haptic_position, haptic_velocity, virtual_objects):
"""Render haptic feedback for the current haptic position."""
total_force = np.zeros(3)
for obj in virtual_objects:
force = self.compute_object_force(haptic_position, haptic_velocity, obj)
total_force += force
# Clamp force for safety
force_magnitude = np.linalg.norm(total_force)
if force_magnitude > self.max_force:
total_force = total_force / force_magnitude * self.max_force
return total_force
def compute_object_force(self, position, velocity, obj):
"""Compute force from a single virtual object."""
if obj['type'] == 'sphere':
return self.sphere_force(position, velocity, obj)
elif obj['type'] == 'plane':
return self.plane_force(position, velocity, obj)
elif obj['type'] == 'mesh':
return self.mesh_force(position, velocity, obj)
return np.zeros(3)
def sphere_force(self, position, velocity, sphere):
"""Force from a sphere object (spring-damper)."""
center = np.array(sphere['center'])
radius = sphere['radius']
# Vector from center to position
delta = position - center
distance = np.linalg.norm(delta)
if distance < radius:
# Inside sphere โ compute penetration
penetration = radius - distance
normal = delta / distance # outward normal
# Spring force (pushes out)
spring_force = self.stiffness * penetration * normal
# Damping force (opposes velocity)
damping_force = -self.damping * np.dot(velocity, normal) * normal
return spring_force + damping_force
return np.zeros(3)
def plane_force(self, position, velocity, plane):
"""Force from a plane (e.g., floor, wall)."""
point = np.array(plane['point'])
normal = np.array(plane['normal'])
normal = normal / np.linalg.norm(normal)
# Signed distance to plane
distance = np.dot(position - point, normal)
if distance < 0:
# Below plane โ penetration
penetration = -distance
# Spring force (pushes up)
spring_force = self.stiffness * penetration * normal
# Friction (opposes tangential velocity)
tangential_velocity = velocity - np.dot(velocity, normal) * normal
friction_force = -plane.get('friction', 0.5) * np.linalg.norm(spring_force) * tangential_velocity
friction_force = np.clip(friction_force, -1, 1) # limit
return spring_force + friction_force
return np.zeros(3)
# Haptic loop at 1kHz
renderer = HapticRenderer()
virtual_objects = [
{'type': 'sphere', 'center': [0.1, 0, 0], 'radius': 0.05},
{'type': 'plane', 'point': [0, -0.1, 0], 'normal': [0, 1, 0], 'friction': 0.6}
]
def haptic_loop():
while True:
position = get_haptic_position()
velocity = get_haptic_velocity()
force = renderer.render(position, velocity, virtual_objects)
set_output_force(force)
time.sleep(0.001) # 1kHz๐งถ Texture & Material Rendering
๐ฏ Feeling surfaces
Texture rendering simulates surface properties: roughness, friction, bumps, and material feel. Techniques: vibrotactile (vibration patterns), electrostatic (friction modulation), ultrasonic (mid-air).
Texture Rendering Methods
Vibrotactile: Vibration frequency/amplitude encodes texture. Electrostatic: Modulates friction between finger and surface. Ultrasonic: Air pressure modulation. Data-driven: Record real textures, play back. Perceptual: Model based on human perception. Materials: Sandpaper, wood, metal, fabric, water.
๐ณ Vibrotactileโก Electrostatic๐ Ultrasonic๐จ Data-driven
# Texture rendering with vibration patterns
import numpy as np
from scipy import signal
class TextureRenderer:
def __init__(self, sample_rate=1000):
self.fs = sample_rate
self.textures = {}
def add_texture(self, name, roughness, friction, bumpiness):
"""Define a texture by its physical properties."""
self.textures[name] = {
'roughness': roughness, # 0-1
'friction': friction, # 0-1
'bumpiness': bumpiness, # 0-1
'pattern': self.generate_pattern(roughness, bumpiness)
}
def generate_pattern(self, roughness, bumpiness):
"""Generate vibration pattern for texture."""
# Roughness -> frequency (higher = rougher)
freq = 20 + roughness * 200 # 20-220 Hz
# Bumpiness -> amplitude modulation
# Use noise for irregular bumps
t = np.linspace(0, 1, self.fs)
# Base vibration
vibration = np.sin(2 * np.pi * freq * t)
# Add noise for bumpiness
noise = np.random.randn(len(t)) * bumpiness
vibration += noise
# Amplitude modulation (roughness variation)
envelope = 1 + 0.5 * np.sin(2 * np.pi * 5 * t) # 5 Hz modulation
vibration *= envelope
return vibration
def render(self, texture_name, velocity):
"""Render texture based on sliding velocity."""
if texture_name not in self.textures:
return np.zeros(self.fs)
texture = self.textures[texture_name]
pattern = texture['pattern']
# Texture intensity depends on sliding velocity
# (faster = more sensation)
speed = np.linalg.norm(velocity)
intensity = min(speed / 0.5, 1.0) # max at 0.5 m/s
# Friction depends on texture
friction_force = texture['friction'] * speed
# Return vibration signal + friction
return {
'vibration': pattern * intensity,
'friction_force': friction_force
}
# Usage
renderer = TextureRenderer()
renderer.add_texture('sandpaper', roughness=0.9, friction=0.8, bumpiness=0.7)
renderer.add_texture('silk', roughness=0.1, friction=0.3, bumpiness=0.05)
renderer.add_texture('wood', roughness=0.5, friction=0.5, bumpiness=0.3)
renderer.add_texture('metal', roughness=0.2, friction=0.2, bumpiness=0.1)
# Render while sliding finger
velocity = np.array([0.3, 0, 0]) # sliding at 0.3 m/s
feedback = renderer.render('sandpaper', velocity)
print(f"Vibration amplitude: {np.max(feedback['vibration']):.2f}")
print(f"Friction force: {feedback['friction_force']:.2f} N")๐ฅฝ Haptics in AR/VR
๐ฏ Completing the immersive experience
Haptics in AR/VR provides touch feedback for virtual objects: vibration when touching, force when grabbing, texture when sliding, temperature when interacting. It dramatically increases presence and immersion.
Haptic Feedback in XR
Controllers: Rumble on contact (Quest, Index, PSVR2). Hand tracking: Ultrasonic mid-air, gloves. Body: Haptic vests, suits (bHaptics, Teslasuit). Weapons: Recoil, reloading. Social: Virtual handshake, hug. Environment: Rain, wind, heat. Presence: Touch increases immersion by 40%+.
๐ฎ Controllers๐งค Gloves๐ฆบ Vests๐ Mid-Air
// Haptic feedback in Unity XR (XR Interaction Toolkit)
using UnityEngine;
using UnityEngine.XR.Interaction.Toolkit;
public class HapticFeedback : MonoBehaviour {
public XRBaseController leftController;
public XRBaseController rightController;
// Haptic presets
public HapticPreset softTouch = new HapticPreset(0.2f, 0.05f);
public HapticPreset hardImpact = new HapticPreset(1.0f, 0.1f);
public HapticPreset textureVibration = new HapticPreset(0.3f, 0.02f);
void OnCollisionEnter(Collision collision) {
// Determine which hand collided
XRBaseController controller = GetControllerForCollider(collision.collider);
if (controller != null) {
// Compute impact force
float impactForce = collision.relativeVelocity.magnitude;
// Select haptic preset based on force
HapticPreset preset = impactForce > 2.0f ? hardImpact : softTouch;
// Play haptic feedback
controller.SendHapticImpulse(preset.amplitude, preset.duration);
}
}
void OnTriggerStay(Collider other) {
// Texture feedback while sliding
if (other.CompareTag("Textured")) {
float velocity = GetHandVelocity();
if (velocity > 0.1f) { // sliding
XRBaseController controller = GetControllerForCollider(other);
controller.SendHapticImpulse(
textureVibration.amplitude * Mathf.Min(velocity, 1.0f),
textureVibration.duration
);
}
}
}
XRBaseController GetControllerForCollider(Collider collider) {
// Determine which hand based on collider position
if (collider.transform.position.x < 0) {
return leftController;
} else {
return rightController;
}
}
}
[System.Serializable]
public struct HapticPreset {
public float amplitude; // 0-1
public float duration; // seconds
public HapticPreset(float amp, float dur) {
amplitude = amp;
duration = dur;
}
}
// Haptic vest control (bHaptics)
using Bhaptics.SDK2;
public class HapticVest : MonoBehaviour {
public string appId = "your-app-id";
public string apiKey = "your-api-key";
void Start() {
BhapticsLibrary.Initialize(apiKey, appId, "https://api.bhaptics.com");
}
public void PlayImpact(Vector3 position, float intensity) {
// Map world position to vest motor
// Vest has 40 motors (8x5 grid front + back)
int motorIndex = WorldToVestMotor(position);
// Play haptic
BhapticsLibrary.PlayMotors((int)PositionType.Vest,
new int[] { motorIndex },
(int)(intensity * 100), // duration ms
100); // intensity 0-100
}
int WorldToVestMotor(Vector3 position) {
// Simplified mapping
float x = Mathf.InverseLerp(-1, 1, position.x); // left-right
float y = Mathf.InverseLerp(0, 2, position.y); // bottom-top
int col = Mathf.FloorToInt(x * 7); // 8 columns
int row = Mathf.FloorToInt(y * 4); // 5 rows
return row * 8 + col;
}
}โ ๏ธ Challenges & Limitations
๐ฏ Why haptics is hard
Haptics faces unique challenges: high update rates, stability, actuator limitations, power consumption, wearability, and the complexity of human touch perception.
Key Challenges
Update rate: 1kHz needed for stable force feedback. Stability: Stiff surfaces cause oscillations. Actuator limits: Force, displacement, bandwidth. Power: Haptic gloves drain batteries fast. Wearability: Bulky, uncomfortable. Perception: Individual differences, adaptation. Safety: Electrical, thermal, mechanical. Cost: High-quality haptics is expensive.
โก 1kHz๐ Stability๐ Power๐งค Wearability
# Stability in haptic rendering
import numpy as np
class StableHapticRenderer:
def __init__(self, stiffness=1000, damping=5, mass=0.1, dt=0.001):
self.k = stiffness
self.b = damping
self.m = mass
self.dt = dt # 1ms
# Stability criterion for virtual coupling
# k * dt^2 / m < 1 (simplified)
self.max_stable_stiffness = mass / (dt ** 2)
print(f"Max stable stiffness: {self.max_stable_stiffness:.0f} N/m")
if stiffness > self.max_stable_stiffness:
print(f"Warning: stiffness {stiffness} exceeds stability limit!")
self.k = self.max_stable_stiffness * 0.9
def render(self, position, velocity, target_position):
"""Render force with stability guarantees."""
# Penetration
penetration = np.linalg.norm(position - target_position)
if penetration > 0:
direction = (position - target_position) / penetration
# Spring force
spring_force = self.k * penetration * direction
# Damping force
damping_force = -self.b * velocity
# Total
force = spring_force + damping_force
# Clamp for safety
max_force = 20.0 # N
force_mag = np.linalg.norm(force)
if force_mag > max_force:
force = force / force_mag * max_force
return force
return np.zeros(3)
def check_passivity(self, energy_history):
"""Check passivity for stability."""
# Passivity: energy must not increase
# E(t) = 0.5 * m * v^2 + 0.5 * k * x^2
# If energy increases, system is active (unstable)
if len(energy_history) < 2:
return True
energy_increase = energy_history[-1] - energy_history[-2]
return energy_increase <= 0
# Power consumption estimation
def estimate_power(actuator_type, force, velocity, duty_cycle):
"""Estimate power consumption of haptic actuator."""
if actuator_type == 'ERM':
# ERM: power proportional to voltage^2
voltage = force / 10.0 # simplified
power = (voltage ** 2) / 10.0 # ohms
return power * duty_cycle
elif actuator_type == 'LRA':
# LRA: more efficient
power = force * velocity * 0.5
return power * duty_cycle
elif actuator_type == 'piezo':
# Piezo: high voltage, low current
power = force * velocity * 0.3
return power * duty_cycle
elif actuator_type == 'voice_coil':
# Voice coil: high power
power = force * velocity * 0.8
return power * duty_cycle
return 0
# Battery life estimation
def battery_life(battery_capacity_mAh, power_consumption_W, voltage=3.7):
"""Estimate battery life in hours."""
energy_Wh = battery_capacity_mAh * voltage / 1000
return energy_Wh / power_consumption_W
# Example: Haptic glove with 5 LRA actuators
power_per_actuator = estimate_power('LRA', force=0.5, velocity=0.1, duty_cycle=0.3)
total_power = power_per_actuator * 5
print(f"Power per actuator: {power_per_actuator:.2f} W")
print(f"Total power: {total_power:.2f} W")
print(f"Battery life (2000mAh): {battery_life(2000, total_power):.1f} hours")๐ Future: Neural Haptics
๐ฏ The future of touch
Neural haptics uses AI to create more realistic, personalized, and efficient touch sensations: neural rendering, predictive haptics, brain-computer interfaces, and soft robotics.
Neural Haptics Advances
Neural rendering: Deep learning generates haptic signals from visual/tactile data. Predictive: Anticipate contact before it happens. Personalized: Adapt to individual perception. BCI: Direct neural stimulation for touch. Soft robotics: Flexible, wearable, natural. Digital twins: Haptic models of real objects. Haptic internet: Touch over the network.
๐ง Neural๐ฎ Predictive๐งฌ BCI๐ค Soft Robotics
# Neural haptic rendering (PyTorch)
import torch
import torch.nn as nn
class NeuralHapticRenderer(nn.Module):
"""Neural network that maps object properties + interaction
to haptic actuator signals."""
def __init__(self, input_dim=10, hidden_dim=128, output_dim=5):
super().__init__()
self.network = nn.Sequential(
nn.Linear(input_dim, hidden_dim),
nn.ReLU(),
nn.Linear(hidden_dim, hidden_dim),
nn.ReLU(),
nn.Linear(hidden_dim, hidden_dim),
nn.ReLU(),
nn.Linear(hidden_dim, output_dim),
nn.Tanh() # output in [-1, 1]
)
def forward(self, x):
# x: [object_stiffness, object_friction, object_roughness,
# penetration, velocity_x, velocity_y, velocity_z,
# user_grip_force, user_skin_temp, previous_haptic]
return self.network(x)
# Training data from real interactions
class HapticDataset(torch.utils.data.Dataset):
def __init__(self, data_path):
# Load recorded data: object properties, interactions, user ratings
self.data = torch.load(data_path)
def __getitem__(self, idx):
return self.data[idx]
# Train neural haptic renderer
model = NeuralHapticRenderer()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
criterion = nn.MSELoss()
for epoch in range(100):
for batch in dataloader:
input_features, target_haptics = batch
# Predict
predicted_haptics = model(input_features)
# Loss
loss = criterion(predicted_haptics, target_haptics)
optimizer.zero_grad()
loss.backward()
optimizer.step()
print(f"Epoch {epoch}: loss = {loss.item():.4f}")
# Inference: predict haptic signals in real-time
def render_haptics(object_props, interaction, user_state):
features = torch.tensor([
object_props['stiffness'],
object_props['friction'],
object_props['roughness'],
interaction['penetration'],
interaction['velocity'][0],
interaction['velocity'][1],
interaction['velocity'][2],
user_state['grip_force'],
user_state['skin_temp'],
0.0 # previous haptic (feedback)
])
with torch.no_grad():
haptic_signals = model(features)
# Map to actuators
actuator_signals = {
'index_vibration': haptic_signals[0].item(),
'middle_vibration': haptic_signals[1].item(),
'ring_vibration': haptic_signals[2].item(),
'pinky_vibration': haptic_signals[3].item(),
'thumb_vibration': haptic_signals[4].item()
}
return actuator_signals
# Example
object_props = {'stiffness': 0.8, 'friction': 0.5, 'roughness': 0.7}
interaction = {'penetration': 0.01, 'velocity': [0.1, 0, 0]}
user_state = {'grip_force': 2.0, 'skin_temp': 32.0}
actuators = render_haptics(object_props, interaction, user_state)
print(actuators)