Guidelines · Spatial AI · Conversational UX

Creative recommendations for embodied, spatial AI agents

There's no standard or state of the art for conversational AI agents that are visible in AR and have a spatial "body." After prototyping and playtesting at Meta Reality Labs, I noticed a consistent lack of empathy and hospitality in spatial agent design. In my own prototypes, I tested strategies for making these experiences warmer and more intuitive, and I codified my lessons learned. These are some of those principles.

Source

Meta Reality Labs

Method

Prototyping · UXR · Playtesting

Scope

15 principles · AR wearables

Status

Abridged · full detail on request

These learnings apply to any AI agent that exists in physical or augmented space — not NPCs in games, but assistive agents that chat, infer context, make smart suggestions, and guide users through experiences. If you want to get into the details, reach out.

  • Set Clear Expectations

    If users expect AI to be infinitely capable, they’ll be disappointed. Build trust by being clear upfront about what the AI can / can’t do.

  • Signal State Visually

    Help users adjust to turn-taking and latency by creating different animations for when AI is listening, thinking, talking, etc.

  • Follow the User's Lead

    Don’t hurry users down a preset progression path — let them explore at their own speed

  • Be Proactive (but not pushy)

    Especially for users new to XR, contextually relevant hints and shortcuts can feel magically helpful and time-saving.

  • Avoid Humanoid Forms

    Human-looking AIs run a high risk of feeling “off” or uncanny. By contrast, non-human embodied AI can help people feel safe and not judged.

  • Remind Users To Keep Talking

    Users new to conversational AI aren’t used to progressing through voice — check in on them when there’s long downtime, and remind them they can just talk.

  • Be Expressive! Morph & Transform

    Match the AI’s movements, expressions, and voice to the mood of the content. It helps users connect and stay engaged.

  • Stay in View

    Centrally anchor the AI when it’s talking — it’s disorienting to hear a disembodied voice. You can also ground its presence with spatial audio.

  • Enable Summoning

    Consider a shortcut that brings the AI back to center — a gesture or word that summons it instantly.

  • Keep It Brief & Interruptible

    The AI should speak in very short chunks of text, and check in as it goes. Long monologues make users feel trapped.

  • Demonstrate Contextual Awareness

    Contextual awareness is an XR AI’s superpower. Flex it right away by commenting on the environment, to pique interest and build trust.

  • Move Through Space

    Exploit the spatial canvas, and bring the AI to life, by moving it around the space naturally and dynamically as it talks and operates.

  • Use Sound Design For "Wow" Moments

    Make key beats in the user journey (like a big reveal, task completion, or fancy trick) feel bigger with sound cues. Otherwise, magic moments can fall flat.

  • Reinforce Voice With Visuals

    When an AI rattles off options or any kind of list, it’s helpful to reinforce voice with visuals — cards, buttons, or even subtitles. This lightens the cognitive load of retaining auditory info.

  • Let Users Customize Character Design

    Given the option, most users prefer to choose the personality of the AI. Set baseline parameters and let users fine-tune.

Next

Spatial Shopping