Currently, the Smart Glasses prototype uses sound detection rather than voice detection. This means the audio input system listens for any speech from the user, rather than specific words. This was a design trade-off to ensure the demonstration was completed in time.
As a result, the experience is best suited to use with a facilitator who can set user expectations for how to speak with the AI.
We have implemented the systems required to update this to true AI-based voice detection in short order, so an upgrade will be a small task.
The AR experience uses Meta’s proprietary colocation system, shared spatial anchors. This system takes a snapshot of the lidar information captured by one headset and shares it with all connected headsets. It is very flexible, but it is an experimental feature. Depending on lighting and environmental factors, the headsets can lose the “lock” on the shared anchor, causing the shared frame of reference between players to become misaligned.
We’ve implemented the “Resync Player” UI element within the AR scene to allow users to easily realign their views.
Long-term, this system can be automated. We can either automatically recall the “sync” method when we detect a player’s frame of reference shifting, recall the “sync” system on a set timer, or move to a more stable QR-code-based tracking system.
As of May 2026, the AR demonstration requires access to the internet to function correctly.