🧠 Cloud Anchors Fundamentals

🎯 What are Cloud Anchors?

Cloud Anchors are persistent, shared anchors stored in the cloud that allow multiple devices to see the same AR content at the same physical location, across sessions and users. They enable multi-user AR, persistent AR, and cross-session experiences. Think of them as "save points" for AR content in the real world.

Why Cloud Anchors Matter

Persistent: Content stays in place across sessions. Shared: Multiple users see the same content. Cross-platform: Works across iOS and Android. Scalable: Millions of anchors. Applications: Multiplayer games, collaborative design, persistent art, location-based experiences. Foundation: The AR Cloud — a shared 3D map of the world.

📌 Persistent📌 Shared📌 Multi-User📌 Cross-Platform
# Cloud Anchors pipeline (conceptual) class CloudAnchorSystem: def __init__(self, provider='arcore'): self.provider = provider self.local_anchors = {} self.cloud_anchors = {} def host_anchor(self, local_anchor): """Host a local anchor to the cloud.""" # 1. Serialize local anchor anchor_data = self.serialize(local_anchor) # 2. Send to cloud cloud_anchor_id = self.cloud_api.host(anchor_data) # 3. Store mapping self.cloud_anchors[cloud_anchor_id] = local_anchor return cloud_anchor_id def resolve_anchor(self, cloud_anchor_id): """Resolve a cloud anchor to a local pose.""" # 1. Request anchor from cloud anchor_data = self.cloud_api.resolve(cloud_anchor_id) # 2. Localize in current environment local_pose = self.localize(anchor_data) # 3. Create local anchor local_anchor = self.create_local_anchor(local_pose) return local_anchor def share_anchor(self, cloud_anchor_id, users): """Share anchor with other users.""" for user in users: self.cloud_api.grant_access(cloud_anchor_id, user.id)

⚙️ How Cloud Anchors Work

🎯 The technical pipeline

Cloud anchors work by: 1. Hosting — capture visual features around a local anchor, upload to cloud. 2. Storing — cloud stores the feature map. 3. Resolving — other devices capture their surroundings, match against the cloud feature map, and compute the anchor's pose.

Hosting & Resolving

Hosting: Device captures feature points around the anchor, uploads to cloud. Cloud processing: Builds a feature map, stores it. Resolving: Device captures its surroundings, sends features to cloud. Matching: Cloud matches features, computes anchor pose. Localization: Device localizes anchor in its own coordinate frame. Quality: Based on feature match quality.

📤 Host☁️ Store📥 Resolve🎯 Localize
# Cloud anchor hosting and resolving (detailed) class CloudAnchorPipeline: def __init__(self, session): self.session = session self.feature_extractor = FeatureExtractor() self.cloud_client = CloudClient() def host(self, local_anchor, ttl_days=365): """Host a local anchor to the cloud.""" # 1. Capture feature map around anchor feature_map = self.capture_features(local_anchor) # 2. Serialize anchor data anchor_data = { 'pose': local_anchor.pose, 'features': feature_map, 'timestamp': time.time(), 'ttl': ttl_days * 86400 } # 3. Upload to cloud response = self.cloud_client.post('/anchors', { 'data': anchor_data, 'api_key': self.api_key }) cloud_anchor_id = response['anchor_id'] # 4. Store mapping self.local_anchors[cloud_anchor_id] = local_anchor return cloud_anchor_id def resolve(self, cloud_anchor_id): """Resolve a cloud anchor to a local pose.""" # 1. Capture current surroundings current_features = self.capture_current_features() # 2. Request anchor from cloud response = self.cloud_client.post(f'/anchors/{cloud_anchor_id}/resolve', { 'features': current_features, 'api_key': self.api_key }) if response['status'] != 'success': return None # 3. Get anchor pose in cloud coordinate frame cloud_pose = response['pose'] # 4. Localize in current environment local_pose = self.localize(cloud_pose, current_features) # 5. Create local anchor local_anchor = self.create_local_anchor(local_pose) return local_anchor def capture_features(self, anchor): """Capture visual features around an anchor.""" # Get camera frames around anchor frames = self.get_frames_around(anchor, radius=1.0) # Extract features features = [] for frame in frames: kp, desc = self.feature_extractor.extract(frame) features.append({ 'keypoints': kp, 'descriptors': desc, 'pose': frame.camera_pose }) return features def localize(self, cloud_pose, current_features): """Localize cloud pose in current environment.""" # Match current features to cloud features matches = self.match_features(current_features, cloud_pose.features) # Compute transformation transformation = self.compute_transformation(matches) # Apply to cloud pose local_pose = transformation @ cloud_pose.pose return local_pose

🤖 ARCore Cloud Anchors

🎯 Google's cloud anchor solution

ARCore Cloud Anchors enable persistent, shared AR experiences on Android and iOS (via ARCore for iOS). They support hosting, resolving, and sharing anchors across devices. Free for developers with usage limits.

ARCore Cloud Anchor API

Host: hostCloudAnchorAsync() — upload anchor. Resolve: resolveCloudAnchorAsync() — download and localize. TTL: 1-365 days. Quotas: Free tier, paid for scale. Quality: Based on feature match quality. Cross-platform: Works with ARKit via ARCore for iOS.

🤖 Android🍎 iOS📤 Host📥 Resolve
// ARCore Cloud Anchors (Kotlin/Android) import com.google.ar.core.Anchor import com.google.ar.core.Anchor.CloudAnchorState import com.google.ar.core.Session import com.google.ar.core.Config class ARCoreCloudAnchors(private val session: Session) { // 1. Configure session for cloud anchors fun configure() { val config = Config(session).apply { cloudAnchorMode = Config.CloudAnchorMode.ENABLED } session.configure(config) } // 2. Host a cloud anchor fun hostAnchor(localAnchor: Anchor): String? { // Host the anchor with TTL (days) val cloudAnchor = session.hostCloudAnchorAsync(localAnchor, 365) // Wait for result var cloudAnchorId: String? = null var state = cloudAnchor.cloudAnchorState while (state == CloudAnchorState.TASK_IN_PROGRESS) { Thread.sleep(100) state = cloudAnchor.cloudAnchorState } if (state == CloudAnchorState.SUCCESS) { cloudAnchorId = cloudAnchor.cloudAnchorId Log.d("CloudAnchor", "Hosted: $cloudAnchorId") } else { Log.e("CloudAnchor", "Host failed: $state") } return cloudAnchorId } // 3. Resolve a cloud anchor fun resolveAnchor(cloudAnchorId: String): Anchor? { // Resolve the cloud anchor val cloudAnchor = session.resolveCloudAnchorAsync(cloudAnchorId) // Wait for result var state = cloudAnchor.cloudAnchorState while (state == CloudAnchorState.TASK_IN_PROGRESS) { Thread.sleep(100) state = cloudAnchor.cloudAnchorState } if (state == CloudAnchorState.SUCCESS) { Log.d("CloudAnchor", "Resolved: $cloudAnchorId") return cloudAnchor } else { Log.e("CloudAnchor", "Resolve failed: $state") return null } } // 4. Cancel hosting/resolving fun cancel(cloudAnchor: Anchor) { session.cancelCloudAnchorAsyncTask(cloudAnchor) } // 5. Cloud anchor states fun handleState(state: CloudAnchorState) { when (state) { CloudAnchorState.NONE -> { /* No state */ } CloudAnchorState.TASK_IN_PROGRESS -> { /* Working... */ } CloudAnchorState.SUCCESS -> { /* Success! */ } CloudAnchorState.ERROR_INTERNAL -> { /* Internal error */ } CloudAnchorState.ERROR_NOT_AUTHORIZED -> { /* Not authorized */ } CloudAnchorState.ERROR_RESOURCE_EXHAUSTED -> { /* Quota exceeded */ } CloudAnchorState.ERROR_HOSTING_DATASET_PROCESSING_FAILED -> { /* Host failed */ } CloudAnchorState.ERROR_CLOUD_ID_NOT_FOUND -> { /* ID not found */ } CloudAnchorState.ERROR_RESOLVING_LOCALIZATION_NO_MATCH -> { /* No match */ } CloudAnchorState.ERROR_RESOLVING_SDK_VERSION_TOO_OLD -> { /* SDK too old */ } CloudAnchorState.ERROR_RESOLVING_SDK_VERSION_TOO_NEW -> { /* SDK too new */ } } } } // ARCore Cloud Anchors in Unity (C#) using Google.XR.ARCoreExtensions; using UnityEngine.XR.ARFoundation; public class ARCoreCloudAnchorsUnity : MonoBehaviour { public ARAnchorManager anchorManager; public ARCloudAnchorManager cloudAnchorManager; // Host anchor public void HostAnchor(ARAnchor localAnchor) { var cloudAnchor = cloudAnchorManager.QueueHostAnchor(localAnchor, 365); // Wait for callback } // Resolve anchor public void ResolveAnchor(string cloudAnchorId) { var cloudAnchor = cloudAnchorManager.QueueResolveAnchor(cloudAnchorId); // Wait for callback } // Callback void OnCloudAnchorStateChanged(ARCloudAnchor cloudAnchor) { if (cloudAnchor.cloudAnchorState == CloudAnchorState.Success) { string id = cloudAnchor.cloudAnchorId; Debug.Log($"Cloud anchor ready: {id}"); } } }

🍎 ARKit & Azure Spatial Anchors

🎯 Cloud anchors on iOS

ARKit doesn't have native cloud anchors, but you can use: ARWorldMap (local persistence), Azure Spatial Anchors (cross-platform cloud), or ARCore Cloud Anchors for iOS. Azure Spatial Anchors is the most robust cross-platform solution.

Azure Spatial Anchors

Cross-platform: iOS, Android, HoloLens. Persistent: Anchors last indefinitely. Shared: Multiple users, multiple devices. Robust: Handles large spaces. Integration: Unity, native iOS/Android. Pricing: Pay per anchor.

☁️ Azure🍎 iOS🤖 Android🥽 HoloLens
// Azure Spatial Anchors (Unity C#) using Microsoft.Azure.SpatialAnchors; using Microsoft.Azure.SpatialAnchors.Unity; public class AzureSpatialAnchors : MonoBehaviour { private CloudSpatialAnchorSession cloudSession; private CloudSpatialAnchor cloudAnchor; async void Start() { // 1. Initialize session cloudSession = new CloudSpatialAnchorSession(); cloudSession.Configuration.AccountId = "your-account-id"; cloudSession.Configuration.AccountKey = "your-account-key"; cloudSession.Configuration.AccountDomain = "your-domain"; cloudSession.SessionUpdated += OnSessionUpdated; cloudSession.AnchorLocated += OnAnchorLocated; cloudSession.LocateAnchorsCompleted += OnLocateCompleted; cloudSession.Start(); } // 2. Create and save anchor async Task CreateAnchor(Vector3 position) { // Create local anchor var localAnchor = new GameObject("Anchor").AddComponent(); localAnchor.transform.position = position; // Create cloud anchor cloudAnchor = new CloudSpatialAnchor(); cloudAnchor.LocalAnchor = localAnchor.NativeAnchor; cloudAnchor.Expiration = DateTimeOffset.Now.AddDays(7); // Save to cloud await cloudSession.CreateAnchorAsync(cloudAnchor); string anchorId = cloudAnchor.Identifier; Debug.Log($"Anchor created: {anchorId}"); } // 3. Locate anchor async Task LocateAnchor(string anchorId) { var criteria = new AnchorLocateCriteria(); criteria.Identifiers = new[] { anchorId }; var watcher = cloudSession.CreateWatcher(criteria); // Wait for AnchorLocated event } // 4. Events void OnSessionUpdated(object sender, SessionUpdatedEventArgs args) { // Session ready } void OnAnchorLocated(object sender, AnchorLocatedEventArgs args) { if (args.Status == LocateAnchorStatus.Located) { // Anchor found var anchor = args.Anchor; // Place content at anchor } } void OnLocateCompleted(object sender, LocateAnchorsCompletedEventArgs args) { // All anchors located } async void OnDestroy() { if (cloudAnchor != null) { await cloudSession.DeleteAnchorAsync(cloudAnchor); } cloudSession.Dispose(); } } // ARKit ARWorldMap (local persistence) /* // Save world map session.getCurrentWorldMap { worldMap, error in let data = try! NSKeyedArchiver.archivedData( withRootObject: worldMap!, requiringSecureCoding: true ) try! data.write(to: worldMapURL) } // Load world map let data = try! Data(contentsOf: worldMapURL) let worldMap = try! NSKeyedUnarchiver.unarchivedObject( ofClass: ARWorldMap.self, from: data ) let config = ARWorldTrackingConfiguration() config.initialWorldMap = worldMap session.run(config) */

🌍 Niantic Lightship VPS

🎯 Visual Positioning System at scale

Niantic Lightship VPS (Visual Positioning System) provides city-scale AR anchoring using a pre-built map of the world. It enables precise localization (cm-level) in outdoor environments, powering Pokémon GO and other location-based AR experiences.

Lightship VPS Features

City-scale: Pre-built maps of cities. Precise: Centimeter-level localization. Persistent: Anchors last indefinitely. Shared: All users see the same content. Outdoor: Optimized for streets, parks. Wayspots: Designated AR locations.

🌍 City-scale📍 cm-level🚶 Outdoor📌 Wayspots
// Niantic Lightship VPS (Unity C#) using Niantic.Lightship.AR.VpsCoverage; using Niantic.Lightship.AR.LocationAR; using Niantic.Lightship.AR.PersistentAnchors; public class LightshipVPS : MonoBehaviour { public ARLocationManager locationManager; public ARPersistentAnchorManager persistentAnchorManager; async void Start() { // 1. Get wayspots near user var wayspots = await CoverageClientFactory.GetWayspotsNearby( latitude, longitude, radius: 1000 ); // 2. Select a wayspot var selectedWayspot = wayspots[0]; // 3. Create AR location var location = new ARLocation { LocationName = selectedWayspot.Name, Latitude = selectedWayspot.Latitude, Longitude = selectedWayspot.Longitude, Altitude = selectedWayspot.Altitude }; // 4. Set location locationManager.SetARLocations(new[] { location }); locationManager.StartTracking(); // 5. Create persistent anchor var anchor = persistentAnchorManager.CreateAnchor( position, rotation ); // 6. Save anchor await persistentAnchorManager.SaveAnchorAsync(anchor); } // Listen for anchor localization void OnEnable() { persistentAnchorManager.PersistentAnchorTrackableAdded += OnAnchorAdded; } void OnAnchorAdded(ARPersistentAnchorEventArgs args) { // Anchor localized var anchor = args.PersistentAnchor; Debug.Log($"Anchor localized: {anchor.trackableId}"); } } // Lightship VPS REST API /* // Get wayspots GET https://api.lightship.dev/v1/wayspots?lat=37.7749&lng=-122.4194&radius=1000 // Response: { "wayspots": [ { "id": "wayspot-123", "name": "Golden Gate Park", "latitude": 37.7694, "longitude": -122.4862, "altitude": 50.0, "image": "https://...", "description": "..." } ] } // Localize POST https://api.lightship.dev/v1/localize { "image": "base64_encoded_image", "wayspot_id": "wayspot-123" } // Response: { "position": {"x": 1.2, "y": 0.5, "z": -3.4}, "rotation": {"x": 0, "y": 0.707, "z": 0, "w": 0.707}, "confidence": 0.95 } */

📤 Hosting an Anchor

🎯 Uploading an anchor to the cloud

Hosting captures the visual features around a local anchor and uploads them to the cloud. The cloud builds a feature map that other devices can match against. Hosting quality depends on: lighting, texture, movement, and angle coverage.

Hosting Best Practices

Good lighting: Even, bright, no shadows. Textured surfaces: Avoid blank walls. Move around: Capture from multiple angles. Stay close: 1-2m from anchor. Slow movement: Avoid motion blur. Stable connection: Wi-Fi recommended. Wait for success: Don't cancel.

💡 Lighting🧶 Texture🔄 Angles📶 Connection
// Hosting an anchor with quality assessment (Unity) using Google.XR.ARCoreExtensions; using UnityEngine.XR.ARFoundation; public class CloudAnchorHost : MonoBehaviour { public ARAnchorManager anchorManager; public ARCloudAnchorManager cloudAnchorManager; public ARCameraManager cameraManager; private ARCloudAnchor cloudAnchor; public void HostAnchorAtPosition(Vector3 position) { // 1. Check hosting quality if (!IsGoodHostingQuality()) { Debug.LogWarning("Poor hosting quality. Move to better lit, textured area."); return; } // 2. Create local anchor var localAnchor = new GameObject("LocalAnchor").AddComponent(); localAnchor.transform.position = position; // 3. Host to cloud cloudAnchor = cloudAnchorManager.QueueHostAnchor(localAnchor, 365); Debug.Log("Hosting... move around the anchor slowly."); } bool IsGoodHostingQuality() { // Check feature point count var featurePoints = FindObjectsOfType(); // (simplified — in practice, check ARPointCloud) // Check lighting if (cameraManager.TryGetIntrinsics(out var intrinsics)) { // Check exposure } // Check motion (device should be still) float motion = Input.acceleration.magnitude; if (motion > 0.5f) { Debug.LogWarning("Device moving too much"); return false; } return true; } // Callback when hosting completes void OnCloudAnchorStateChanged(ARCloudAnchor anchor) { switch (anchor.cloudAnchorState) { case CloudAnchorState.Success: string anchorId = anchor.cloudAnchorId; Debug.Log($"Hosted successfully! ID: {anchorId}"); // Share anchorId with other users ShareAnchor(anchorId); break; case CloudAnchorState.TaskInProgress: Debug.Log("Hosting in progress..."); break; case CloudAnchorState.ErrorHostingDatasetProcessingFailed: Debug.LogError("Hosting failed: dataset processing"); break; case CloudAnchorState.ErrorResourceExhausted: Debug.LogError("Hosting failed: quota exceeded"); break; default: Debug.LogError($"Hosting failed: {anchor.cloudAnchorState}"); break; } } void ShareAnchor(string anchorId) { // Share via deep link, QR code, or backend string deepLink = $"myapp://anchor/{anchorId}"; Debug.Log($"Share this link: {deepLink}"); // Or send to backend StartCoroutine(PostAnchorToBackend(anchorId)); } } // Hosting quality indicators public class HostingQualityUI : MonoBehaviour { public UnityEngine.UI.Image qualityIndicator; public TMPro.TextMeshProUGUI qualityText; void Update() { float quality = AssessQuality(); if (quality > 0.8f) { qualityIndicator.color = Color.green; qualityText.text = "Excellent"; } else if (quality > 0.6f) { qualityIndicator.color = Color.yellow; qualityText.text = "Good"; } else if (quality > 0.4f) { qualityIndicator.color = new Color(1, 0.5f, 0); qualityText.text = "Fair"; } else { qualityIndicator.color = Color.red; qualityText.text = "Poor"; } } float AssessQuality() { float score = 1.0f; // Feature point density // Lighting // Motion // Distance return score; } }

📥 Resolving an Anchor

🎯 Finding a cloud anchor

Resolving downloads the anchor's feature map from the cloud, matches it against the device's current view, and localizes the anchor in the device's coordinate frame. Resolution quality depends on visual overlap with the hosting environment.

Resolving Best Practices

Same environment: Resolve where anchor was hosted. Similar lighting: Match hosting conditions. Move around: Capture from multiple angles. Wait: Resolution takes 5-30 seconds. Check quality: Use tracking quality. Retry: If failed, move and try again.

📍 Same Place💡 Same Light🔄 Multiple Angles⏱️ Wait
// Resolving a cloud anchor (Unity) using Google.XR.ARCoreExtensions; using UnityEngine.XR.ARFoundation; public class CloudAnchorResolver : MonoBehaviour { public ARAnchorManager anchorManager; public ARCloudAnchorManager cloudAnchorManager; private ARCloudAnchor cloudAnchor; public void ResolveAnchor(string cloudAnchorId) { Debug.Log($"Resolving anchor: {cloudAnchorId}"); // Resolve from cloud cloudAnchor = cloudAnchorManager.QueueResolveAnchor(cloudAnchorId); // Show UI: "Look around the area where the anchor was placed" ShowResolutionUI(); } void OnCloudAnchorStateChanged(ARCloudAnchor anchor) { switch (anchor.cloudAnchorState) { case CloudAnchorState.Success: Debug.Log("Anchor resolved successfully!"); HideResolutionUI(); PlaceContentAtAnchor(anchor); break; case CloudAnchorState.TaskInProgress: UpdateResolutionProgress(); break; case CloudAnchorState.ErrorResolvingLocalizationNoMatch: Debug.LogError("No match found. Move to the anchor's location."); ShowRetryUI(); break; case CloudAnchorState.ErrorCloudIdNotFound: Debug.LogError("Anchor not found. Check the ID."); ShowErrorUI("Anchor not found"); break; case CloudAnchorState.ErrorResolvingSdkVersionTooOld: Debug.LogError("SDK too old. Please update."); break; default: Debug.LogError($"Resolution failed: {anchor.cloudAnchorState}"); ShowRetryUI(); break; } } void PlaceContentAtAnchor(ARCloudAnchor anchor) { // Place AR content at the resolved anchor var content = Instantiate(contentPrefab, anchor.transform); content.transform.localPosition = Vector3.zero; content.transform.localRotation = Quaternion.identity; } void ShowResolutionUI() { // Show "Look around to find the anchor" // Show progress indicator } void UpdateResolutionProgress() { // Update progress bar } void ShowRetryUI() { // Show "Try again" button } void ShowErrorUI(string message) { // Show error message } } // Resolution quality feedback public class ResolutionQuality : MonoBehaviour { public ARCloudAnchor cloudAnchor; public UnityEngine.UI.Slider qualitySlider; void Update() { if (cloudAnchor == null) return; // Quality is 0-1 float quality = GetResolutionQuality(); qualitySlider.value = quality; if (quality > 0.8f) { // Excellent — content is stable } else if (quality > 0.5f) { // Good — content may drift slightly } else { // Poor — content may be unstable } } float GetResolutionQuality() { // Based on feature match quality // Higher = more features matched = better localization return 0.85f; } } // Multi-anchor resolution public class MultiAnchorResolver : MonoBehaviour { public ARCloudAnchorManager cloudAnchorManager; private Dictionary anchors = new Dictionary(); public void ResolveMultipleAnchors(string[] anchorIds) { foreach (var id in anchorIds) { var anchor = cloudAnchorManager.QueueResolveAnchor(id); anchors[id] = anchor; } } void OnCloudAnchorStateChanged(ARCloudAnchor anchor) { if (anchor.cloudAnchorState == CloudAnchorState.Success) { // Find which anchor this is foreach (var kvp in anchors) { if (kvp.Value == anchor) { Debug.Log($"Anchor {kvp.Key} resolved"); PlaceContentForAnchor(kvp.Key, anchor); } } } } void PlaceContentForAnchor(string anchorId, ARCloudAnchor anchor) { // Load content specific to this anchor var content = LoadContentForAnchor(anchorId); Instantiate(content, anchor.transform); } }

💾 Persistence & Sharing

🎯 Anchors that last

Cloud anchors persist across sessions and can be shared between users. Persistence options: TTL-based (time-limited), permanent (never expires), and user-managed (delete when done). Sharing options: deep links, QR codes, backend, and nearby.

Sharing Methods

Deep link: myapp://anchor/abc123. QR code: Scan to open. Backend: Store in database, query by location. Nearby: Bluetooth/Wi-Fi Direct. Social: Share via messaging apps. Geospatial: Anchor to GPS coordinates.

🔗 Deep Link📱 QR Code🗄️ Backend📍 GPS
// Persistence and sharing system using System; using System.Collections.Generic; using UnityEngine; using UnityEngine.Networking; public class AnchorPersistence : MonoBehaviour { [Serializable] public class AnchorRecord { public string anchorId; public string userId; public string name; public string description; public double latitude; public double longitude; public double altitude; public long createdAt; public long expiresAt; public string contentType; public string contentData; } private string backendUrl = "https://api.myapp.com/anchors"; // 1. Save anchor to backend public IEnumerator SaveAnchor(AnchorRecord record) { string json = JsonUtility.ToJson(record); using (UnityWebRequest request = UnityWebRequest.Post(backendUrl, json)) { request.SetRequestHeader("Content-Type", "application/json"); request.SetRequestHeader("Authorization", $"Bearer {GetAuthToken()}"); yield return request.SendWebRequest(); if (request.result == UnityWebRequest.Result.Success) { Debug.Log("Anchor saved to backend"); } else { Debug.LogError($"Failed to save: {request.error}"); } } } // 2. Load anchors near location public IEnumerator LoadAnchorsNearby(double lat, double lng, float radius) { string url = $"{backendUrl}?lat={lat}&lng={lng}&radius={radius}"; using (UnityWebRequest request = UnityWebRequest.Get(url)) { request.SetRequestHeader("Authorization", $"Bearer {GetAuthToken()}"); yield return request.SendWebRequest(); if (request.result == UnityWebRequest.Result.Success) { var anchors = JsonUtility.FromJson(request.downloadHandler.text); Debug.Log($"Found {anchors.anchors.Length} anchors nearby"); } } } // 3. Generate share link public string GenerateShareLink(string anchorId) { return $"https://myapp.com/anchor/{anchorId}"; } // 4. Generate QR code public Texture2D GenerateQRCode(string anchorId) { string link = GenerateShareLink(anchorId); return QRCodeGenerator.Generate(link); } // 5. Deep link handling void OnEnable() { Application.deepLinkActivated += OnDeepLinkActivated; if (!string.IsNullOrEmpty(Application.absoluteURL)) { OnDeepLinkActivated(Application.absoluteURL); } } void OnDeepLinkActivated(string url) { // Parse anchor ID from URL // myapp://anchor/abc123 if (url.Contains("/anchor/")) { string anchorId = url.Split('/').Last(); ResolveAnchor(anchorId); } } // 6. GPS-based anchor discovery public void FindAnchorsAtLocation(double lat, double lng) { StartCoroutine(LoadAnchorsNearby(lat, lng, 100f)); } string GetAuthToken() { return PlayerPrefs.GetString("auth_token", ""); } } [Serializable] public class AnchorList { public AnchorRecord[] anchors; } // Anchor TTL management public class AnchorTTL : MonoBehaviour { public void SetTTL(string anchorId, int days) { // Update TTL in cloud // TTL: 1-365 days for ARCore } public void ExtendTTL(string anchorId, int additionalDays) { // Extend existing TTL } public void DeleteAnchor(string anchorId) { // Delete from cloud } }

👥 Multi-User AR

🎯 Shared AR experiences

Cloud anchors enable multi-user AR: multiple devices see the same AR content at the same physical location. Combine with networking (Photon, Mirror, Netcode) to sync interactions.

Multi-User Architecture

Host: One user hosts anchor. Share: Anchor ID shared with others. Resolve: Others resolve anchor. Sync: Network interactions. State: Shared game state. Frameworks: Photon, Mirror, Netcode.

📤 Host📥 Resolve🔄 Sync🌐 Network
// Multi-user AR with cloud anchors and Photon using Photon.Pun; using Google.XR.ARCoreExtensions; using UnityEngine.XR.ARFoundation; public class MultiUserAR : MonoBehaviourPunCallbacks { public ARCloudAnchorManager cloudAnchorManager; public ARAnchorManager anchorManager; private string currentAnchorId; // 1. Host creates anchor and shares ID public void HostAnchor(Vector3 position) { // Create local anchor var localAnchor = new GameObject("LocalAnchor").AddComponent(); localAnchor.transform.position = position; // Host to cloud var cloudAnchor = cloudAnchorManager.QueueHostAnchor(localAnchor, 365); // When hosted, share ID via Photon OnAnchorHosted(cloudAnchor); } void OnAnchorHosted(ARCloudAnchor anchor) { if (anchor.cloudAnchorState == CloudAnchorState.Success) { currentAnchorId = anchor.cloudAnchorId; // Share with all players photonView.RPC("OnAnchorIdReceived", RpcTarget.Others, currentAnchorId); } } // 2. Clients resolve anchor [PunRPC] void OnAnchorIdReceived(string anchorId) { currentAnchorId = anchorId; // Resolve anchor var cloudAnchor = cloudAnchorManager.QueueResolveAnchor(anchorId); // When resolved, place content OnAnchorResolved(cloudAnchor); } void OnAnchorResolved(ARCloudAnchor anchor) { if (anchor.cloudAnchorState == CloudAnchorState.Success) { // Place shared content at anchor if (PhotonNetwork.IsMasterClient) { // Master spawns content PhotonNetwork.Instantiate("SharedContent", anchor.transform.position, anchor.transform.rotation); } } } // 3. Sync interactions public void OnObjectInteracted(GameObject obj) { if (photonView.IsMine) { photonView.RPC("SyncInteraction", RpcTarget.Others, obj.name, obj.transform.position); } } [PunRPC] void SyncInteraction(string objName, Vector3 position) { var obj = GameObject.Find(objName); if (obj != null) { obj.transform.position = position; } } } // Multi-user AR with Unity Netcode using Unity.Netcode; using Google.XR.ARCoreExtensions; public class NetcodeAR : NetworkBehaviour { public ARCloudAnchorManager cloudAnchorManager; private NetworkVariable anchorId = new NetworkVariable(); public override void OnNetworkSpawn() { if (IsServer) { // Server hosts anchor HostAnchor(); } else { // Client resolves anchor anchorId.OnValueChanged += OnAnchorIdChanged; } } void HostAnchor() { var localAnchor = new GameObject("Anchor").AddComponent(); var cloudAnchor = cloudAnchorManager.QueueHostAnchor(localAnchor, 365); // When hosted, set network variable OnAnchorHosted(cloudAnchor); } void OnAnchorHosted(ARCloudAnchor anchor) { if (anchor.cloudAnchorState == CloudAnchorState.Success) { anchorId.Value = anchor.cloudAnchorId; } } void OnAnchorIdChanged(Unity.Collections.FixedString128Bytes oldValue, Unity.Collections.FixedString128Bytes newValue) { // Resolve anchor var cloudAnchor = cloudAnchorManager.QueueResolveAnchor(newValue.ToString()); // Place content when resolved } }

🗺️ Geospatial Anchors

🎯 Anchors tied to GPS

Geospatial anchors combine cloud anchors with GPS coordinates, enabling location-based AR at scale. ARCore Geospatial API, Niantic Lightship VPS, and Google's Visual Positioning System (VPS) provide city-scale localization.

Geospatial AR Features

GPS + VPS: Outdoor localization. City-scale: Works across cities. Persistent: Anchors last indefinitely. Shared: All users see content. Use cases: Tourism, gaming, navigation, advertising. Accuracy: 1-10 meters (GPS) to cm-level (VPS).

📍 GPS🌍 City-scale🚶 Outdoor🎮 Pokémon GO
// ARCore Geospatial API (Kotlin/Android) import com.google.ar.core.GeospatialPose import com.google.ar.core.Session import com.google.ar.core.Earth import com.google.ar.core.Anchor class GeospatialAR(private val session: Session) { fun createGeospatialAnchor(latitude: Double, longitude: Double, altitude: Double) { val earth = session.earth if (earth.trackingState == TrackingState.TRACKING) { // Create anchor at GPS coordinates val anchor = earth.createAnchor( latitude, longitude, altitude, quaternion // orientation ) // Place content placeContent(anchor) } } fun getCameraGeospatialPose(): GeospatialPose? { val earth = session.earth if (earth.trackingState == TrackingState.TRACKING) { return earth.cameraGeospatialPose } return null } fun checkVpsAvailability(latitude: Double, longitude: Double): Boolean { return session.checkVpsAvailability(latitude, longitude) } } // ARCore Geospatial API (Unity) using Google.XR.ARCoreExtensions; using UnityEngine; public class GeospatialARUnity : MonoBehaviour { public AREarthManager earthManager; public ARAnchorManager anchorManager; void Update() { // Check Earth tracking state if (earthManager.EarthTrackingState != TrackingState.Tracking) { return; } // Get camera geospatial pose var cameraPose = earthManager.CameraGeospatialPose; double latitude = cameraPose.Latitude; double longitude = cameraPose.Longitude; double altitude = cameraPose.Altitude; Debug.Log($"Position: {latitude}, {longitude}, {altitude}"); } public void CreateAnchor(double lat, double lng, double alt) { // Check VPS availability var availability = earthManager.CheckVpsAvailability(lat, lng); if (availability != VpsAvailability.Available) { Debug.LogWarning("VPS not available here"); return; } // Create geospatial anchor var anchor = anchorManager.AddAnchor( lat, lng, alt, Quaternion.identity ); // Place content Instantiate(contentPrefab, anchor.transform); } } // Niantic Lightship VPS (Unity) using Niantic.Lightship.AR.VpsCoverage; using Niantic.Lightship.AR.LocationAR; public class LightshipVPSUnity : MonoBehaviour { public ARLocationManager locationManager; async void Start() { // Get wayspots near user var wayspots = await CoverageClientFactory.GetWayspotsNearby( latitude, longitude, radius: 1000 ); // Create AR location from wayspot var location = new ARLocation { LocationName = wayspots[0].Name, Latitude = wayspots[0].Latitude, Longitude = wayspots[0].Longitude, Altitude = wayspots[0].Altitude }; locationManager.SetARLocations(new[] { location }); locationManager.StartTracking(); } }

🎯 Accuracy & Quality

🎯 How good is the localization?

Cloud anchor accuracy depends on: hosting quality, resolving quality, environment, and time since hosting. Typical accuracy: 1-10cm for good conditions, worse for poor conditions.

Factors Affecting Accuracy

Features: More texture = better. Lighting: Even light = better. Viewpoint: Similar angle = better. Time: Longer since hosting = worse. Environment: Static scenes = better. Device: Better sensors = better.

🧶 Features💡 Lighting📐 Viewpoint⏱️ Time
// Accuracy and quality metrics public class AnchorQuality { public float FeatureMatchQuality; // 0-1 public float LocalizationConfidence; // 0-1 public float EstimatedAccuracy; // meters public int FeaturePointsMatched; public int FeaturePointsTotal; public string GetQualityLabel() { if (FeatureMatchQuality > 0.8f) return "Excellent"; if (FeatureMatchQuality > 0.6f) return "Good"; if (FeatureMatchQuality > 0.4f) return "Fair"; return "Poor"; } public Color GetQualityColor() { if (FeatureMatchQuality > 0.8f) return Color.green; if (FeatureMatchQuality > 0.6f) return Color.yellow; if (FeatureMatchQuality > 0.4f) return new Color(1, 0.5f, 0); return Color.red; } } // Accuracy visualization public class AccuracyVisualizer : MonoBehaviour { public ARCloudAnchor anchor; public GameObject accuracySphere; void Update() { if (anchor == null) return; // Get accuracy from anchor float accuracy = GetAnchorAccuracy(anchor); // Scale sphere to accuracy accuracySphere.transform.localScale = Vector3.one * accuracy * 2; // Color by accuracy var renderer = accuracySphere.GetComponent(); if (accuracy < 0.1f) { renderer.material.color = Color.green; } else if (accuracy < 0.5f) { renderer.material.color = Color.yellow; } else { renderer.material.color = Color.red; } } float GetAnchorAccuracy(ARCloudAnchor anchor) { // ARCore provides accuracy estimate // Typically 0.01-1.0 meters return 0.15f; } } // Improving accuracy public class AccuracyImprover : MonoBehaviour { // 1. Host with good coverage public void HostWithCoverage(Vector3 position) { // Move around anchor to capture features // Wait for "excellent" quality before hosting } // 2. Resolve with multiple attempts public void ResolveWithRetry(string anchorId, int maxRetries = 3) { StartCoroutine(ResolveWithRetryCoroutine(anchorId, maxRetries)); } IEnumerator ResolveWithRetryCoroutine(string anchorId, int maxRetries) { for (int i = 0; i < maxRetries; i++) { var anchor = ResolveAnchor(anchorId); // Wait for result yield return new WaitUntil(() => anchor.cloudAnchorState != CloudAnchorState.TaskInProgress); if (anchor.cloudAnchorState == CloudAnchorState.Success) { // Check accuracy if (GetAnchorAccuracy(anchor) < 0.2f) { yield break; // Good enough } } // Move to different angle and retry yield return new WaitForSeconds(1.0f); } } // 3. Use quality indicators public void ShowQualityFeedback(float quality) { if (quality < 0.4f) { ShowMessage("Move to a more textured area"); } else if (quality < 0.6f) { ShowMessage("Move closer to the anchor location"); } else if (quality < 0.8f) { ShowMessage("Good — hold steady"); } else { ShowMessage("Excellent — hosting now"); } } }

🔒 Privacy & Security

🎯 Protecting user data

Cloud anchors capture visual features of the environment — this is sensitive data. Privacy considerations: consent, data retention, encryption, and access control.

Privacy Considerations

Visual data: Feature maps capture environment. Location: GPS coordinates are sensitive. Consent: Users must agree to hosting. Retention: How long are anchors stored? Encryption: Data in transit and at rest. Access: Who can resolve anchors? Regulations: GDPR, CCPA, BIPA.

🔐 Encryption📝 Consent⏱️ Retention⚖️ GDPR
// Privacy-preserving cloud anchors public class PrivacyCloudAnchors { // 1. User consent public async Task RequestConsent() { // Show consent dialog return await ShowConsentDialog( "This app uses cloud anchors to save AR content. " + "It captures visual features of your environment. " + "Do you consent?" ); } // 2. Anonymize anchor data public AnchorRecord AnonymizeAnchor(AnchorRecord record) { // Remove user ID record.userId = null; // Round GPS to ~100m record.latitude = Math.Round(record.latitude, 3); record.longitude = Math.Round(record.longitude, 3); // Remove description record.description = null; return record; } // 3. Encrypt data public byte[] EncryptAnchorData(AnchorRecord record) { var json = JsonUtility.ToJson(record); return AesEncrypt(json, GetEncryptionKey()); } // 4. Access control public bool CanResolveAnchor(string anchorId, string userId) { // Check if user has access var anchor = GetAnchor(anchorId); return anchor.isPublic || anchor.allowedUsers.Contains(userId); } // 5. Data retention public void SetRetentionPolicy(string anchorId, int days) { // Set TTL // Auto-delete after expiry } // 6. Delete user data public void DeleteUserData(string userId) { // Delete all anchors owned by user var userAnchors = GetAnchorsByUser(userId); foreach (var anchor in userAnchors) { DeleteAnchor(anchor.id); } } } // GDPR compliance public class GDPRCompliance { public void HandleDataRequest(string userId, DataRequestType type) { switch (type) { case DataRequestType.Access: ExportUserData(userId); break; case DataRequestType.Deletion: DeleteUserData(userId); break; case DataRequestType.Portability: ExportUserData(userId, format: "json"); break; } } public void ExportUserData(string userId, string format = "json") { var anchors = GetAnchorsByUser(userId); // Export as JSON/CSV } public void DeleteUserData(string userId) { // Delete all anchors // Delete all location history // Delete all visual data } } // Security best practices public class SecurityBestPractices { void Start() { // 1. Use HTTPS for all API calls // 2. Authenticate users // 3. Authorize anchor access // 4. Encrypt sensitive data // 5. Validate all inputs // 6. Rate limit API calls // 7. Monitor for abuse // 8. Regular security audits } }

⚠️ Challenges & Limitations

🎯 What can go wrong

Cloud anchors face challenges: resolution failures, drift over time, environment changes, lighting differences, network dependency, and cost at scale.

Common Issues

Resolution failure: Can't find anchor. Drift: Anchor moves over time. Environment change: Furniture moved, lighting changed. Network: Requires internet. Cost: Hosting/resolving costs money. Privacy: Capturing environment. Latency: Resolution takes time.

❌ Failure📉 Drift🌐 Network💰 Cost
// Handling cloud anchor failures public class CloudAnchorFailureHandler : MonoBehaviour { public ARCloudAnchorManager cloudAnchorManager; // 1. Resolution failure handling public void HandleResolutionFailure(string anchorId, CloudAnchorState state) { switch (state) { case CloudAnchorState.ErrorResolvingLocalizationNoMatch: ShowMessage("Couldn't find the anchor. Move to the anchor location and try again."); ShowRetryButton(anchorId); break; case CloudAnchorState.ErrorCloudIdNotFound: ShowMessage("This anchor no longer exists."); break; case CloudAnchorState.ErrorResourceExhausted: ShowMessage("Too many requests. Please wait and try again."); break; case CloudAnchorState.ErrorNotAuthorized: ShowMessage("You don't have permission to access this anchor."); break; default: ShowMessage($"Resolution failed: {state}"); ShowRetryButton(anchorId); break; } } // 2. Drift detection public bool DetectDrift(ARCloudAnchor anchor, Vector3 expectedPosition) { float drift = Vector3.Distance(anchor.transform.position, expectedPosition); if (drift > 0.5f) { // 50cm drift Debug.LogWarning($"Anchor drifted {drift}m"); return true; } return false; } // 3. Network failure handling public async Task CheckNetwork() { if (Application.internetReachability == NetworkReachability.NotReachable) { ShowMessage("No internet. Cloud anchors require a connection."); return false; } return true; } // 4. Offline fallback public void UseLocalAnchorFallback(Vector3 position) { // Create local anchor if cloud fails var localAnchor = new GameObject("LocalAnchor").AddComponent(); localAnchor.transform.position = position; ShowMessage("Using local anchor (cloud unavailable)"); } // 5. Cost management public void ManageCosts() { // Cache resolved anchors locally // Limit hosting frequency // Use local anchors when possible // Monitor usage } } // Best practices for reliability public class CloudAnchorReliability { void Start() { // 1. Host in good conditions // 2. Test resolution before sharing // 3. Use multiple anchors for redundancy // 4. Handle failures gracefully // 5. Provide offline fallback // 6. Monitor anchor health // 7. Clean up old anchors // 8. Cache resolved anchors } // Redundant anchors public void CreateRedundantAnchors(Vector3 position) { // Create 3 anchors nearby for (int i = 0; i < 3; i++) { Vector3 offset = new Vector3( UnityEngine.Random.Range(-0.5f, 0.5f), 0, UnityEngine.Random.Range(-0.5f, 0.5f) ); HostAnchor(position + offset); } } // Anchor health monitoring public void MonitorAnchorHealth(string anchorId) { // Periodically resolve anchor // Check accuracy // Re-host if degraded } }

🚀 Future: The AR Cloud

🎯 A shared 3D map of the world

The AR Cloud is the ultimate vision: a persistent, shared, 3D map of the world that all AR devices can use. Cloud anchors are the foundation. Future: cm-level accuracy, city-scale, real-time, AI-powered.

The AR Cloud Vision

Persistent: Content stays forever. Shared: Everyone sees the same. Scalable: Entire world mapped. Intelligent: AI understands the scene. Multi-user: Millions of concurrent users. Cross-platform: Any device. Applications: Metaverse, smart cities, autonomous vehicles.

🌍 World Map👥 Shared🧠 AI🚀 Metaverse
// Future: AR Cloud architecture public class ARCloud { // 1. Visual Positioning System (VPS) public async Task Localize(byte[] image, GeoLocation hint) { // Send image to cloud // Match against world map // Return 6-DOF pose return await CloudAPI.Localize(image, hint); } // 2. Persistent world map public async Task GetWorldMap(Bounds bounds) { // Download 3D map for area return await CloudAPI.GetWorldMap(bounds); } // 3. Multi-user sync public void SyncWithUsers(List users) { // Sync state with nearby users // Share anchors, content, interactions } // 4. AI-powered understanding public SceneGraph UnderstandScene(byte[] image) { // Semantic understanding // Object detection // Scene graph return AI.Analyze(image); } // 5. Real-time updates public void UpdateWorldMap(WorldMapDelta delta) { // Upload changes to world map // Merge with existing map } } // Future: Neural AR Cloud public class NeuralARCloud { // 1. NeRF-based world map public NeRF GetNeRF(Bounds bounds) { // Download NeRF of area return CloudAPI.GetNeRF(bounds); } // 2. Gaussian Splatting map public GaussianSplat GetSplat(Bounds bounds) { // Download Gaussian Splat return CloudAPI.GetSplat(bounds); } // 3. Real-time neural rendering public Texture2D Render(Pose pose) { // Render from neural map return NeuralRenderer.Render(pose); } // 4. Generative AR public GameObject GenerateContent(string prompt) { // Generate 3D content from text return TextTo3D.Generate(prompt); } } // Future: AR Cloud platforms /* Meta AR Cloud: - Persistent anchors - Shared experiences - Cross-device Google ARCore Geospatial: - VPS localization - City-scale - Street View data Niantic Lightship: - City-scale VPS - Wayspots - Pokémon GO scale Apple: - ARKit + ARWorldMap - Vision Pro integration - Location Anchors Snap: - Custom Landmarkers - City-scale AR - Lens Studio integration */ // Future: Metaverse integration public class MetaverseAR { public void EnterMetaverse() { // AR Cloud → Metaverse // Persistent virtual world // Shared with millions // Cross-platform } } // Future: Digital twins public class DigitalTwinAR { public void CreateDigitalTwin(Building building) { // Scan building // Create digital twin // Sync with cloud // Overlay AR data } }