CREATIVE RECOMMENDATIONS

for embodied, spatial AI agents

Bit of a mouthful? All I mean is: I prototype a lot of conversational AI agents that are visible in AR, that have a “body” of sorts. There’s no standard or state of the art for this yet. And so far, I’m noticing a real lack of empathy and hospitality in spatial agent design. So I decided to codify and share some of my lessons learned, to help things get better, faster. A few caveats —

Definitions: I’m talking about AI entities that exist to assist, not non-playable characters (aka NPCs) used to populate game worlds and facilitate interaction. These agents are here to help — they chat, infer context, make smart suggestions, and guide users through content experiences.

Evidence: These learnings come from prototypes and playtesting, plus a few UXR rounds.

Approach: This guidance is extremely simplified — but I could go on. If you want to get into the details with me, reach out!

  • Set Clear Expectations

    If users expect AI to be infinitely capable, they’ll be disappointed. Build trust by being clear upfront about what the AI can / can’t do.

  • Signal State Visually

    Help users adjust to turn-taking and latency by creating different animations for when AI is listening, thinking, talking, etc.

  • Follow the User's Lead

    Don’t hurry users down a preset progression path — let them explore at their own speed

  • Be Proactive (but not pushy)

    Especially for users new to XR, contextually relevant hints and shortcuts can feel magically helpful and time-saving.

  • Avoid Humanoid Forms

    Human-looking AIs run a high risk of feeling “off” or uncanny. By contrast, non-human embodied AI can help people feel safe and not judged.

  • Remind Users To Keep Talking

    Users new to conversational AI aren’t used to progressing through voice — check in on them when there’s long downtime, and remind them they can just talk.

  • Be Expressive! Morph & Transform

    Match the AI’s movements, expressions, and voice to the mood of the content. It helps users connect and stay engaged.

  • Stay in View

    Centrally anchor the AI when it’s talking — it’s disorienting to hear a disembodied voice. You can also ground its presence with spatial audio.

  • Enable Summoning

    Consider a shortcut that brings the AI back to center — a gesture or word that summons it instantly.

  • Keep It Brief & Interruptible

    The AI should speak in very short chunks of text, and check in as it goes. Long monologues make users feel trapped.

  • Demonstrate Contextual Awareness

    Contextual awareness is an XR AI’s superpower. Flex it right away by commenting on the environment, to pique interest and build trust.

  • Move Through Space

    Exploit the spatial canvas, and bring the AI to life, by moving it around the space naturally and dynamically as it talks and operates.

  • Use Sound Design For "Wow" Moments

    Make key beats in the user journey (like a big reveal, task completion, or fancy trick) feel bigger with sound cues. Otherwise, magic moments can fall flat.

  • Reinforce Voice With Visuals

    When an AI rattles off options or any kind of list, it’s helpful to reinforce voice with visuals — cards, buttons, or even subtitles. This lightens the cognitive load of retaining auditory info.

  • Let Users Customize Character Design

    Given the option, most users prefer to choose the personality of the AI. Set baseline parameters and let users fine-tune.

Next

Spatial Shopping