Considerations for Experience-wide Interactions
Once you’ve presented the key interaction affordances and helped users to develop a clear mental model of the experience within the first 60 seconds, that mental model of how to interact with the system and what to expect helps them to succeed throughout the rest of the experience.
But what shapes the rest of the experience? How do you decide what its many interactions will be? Next, let’s talk a bit about user agency, interaction capabilities, and virtual personas.
Agency is the empowerment of users to impact their experience—users’ ability to influence the world around them. It can be either global or local.
With global agency, you are giving users the power to create impact that affects the entire experience, regardless of a particular scene. Local agency empowers users to take action in a way that impacts a particular scene or area they currently inhabit within your XR world and is characterized by a more limited sense of power.
In planning your XR experience, it is important to determine how much agency you would like to afford users during each phase of the experience. But remember, agency is power. It has a great influence on the dynamic between users and the extended-reality world they inhabit. So, the XR designer should carefully distribute that power in a way that helps the intended experience unfold.
More power isn’t always better. The right amount of power depends on the type of relationship you want to create between the user and the XR environment. While empowerment is a wonderful capability that highlights the creative wonders that often characterize XR experiences, sometimes restriction is the better way to create the user-to-environment dynamic that best supports the experience you are designing.
Levels of User Agency
One option is to remove user agency altogether. This lets you take users on a directed path, controlling their view, and not allowing any interactive capabilities that could alter the scene. If the goal of your experience would benefit from being directive rather than empowering, this level of user agency could be a viable option. Generally, however, this is not the type of exciting, immersive experience that makes XR enjoyable. Nevertheless, it has its place, depending on the XR designer’s intentions for the experience.
Moving beyond a directed experience, you can opt to give users varying amounts of local agency, empowering them to affect the scene they are in. The level of local agency can vary from the ability to interact with the scene in minor ways such as moving around, to being able to drastically alter the environment around them by picking things up and moving them. However, local agency, in contrast to global agency, is always constrained by scene-level user power.
Beyond local agency, you have the option of giving users global agency. This may entail users’ being able to affect not only individual scenes that they currently inhabit but also parts of the experience that they’ll experience later. This can lead to users’ being able to create their own unique journey for the experience.
The level of agency you provide the user depends on the experience you are trying to achieve—in terms of its being directive versus enabling freedom—and how empowerment plays a role in the design of your local and global experiential intentions.
Decreasing agency sets the stage for more directive or suggestive experiences, while increasing agency also increases user freedom and choice. Of course, the latter implies many potential paths within the experience, so carefully consider the design and development effort that this increased depth of experience would require.
Interaction Capabilities and Virtual Personas
Once you’ve decided the appropriate level of agency, the question then becomes how to go about giving users the ability to take agency. Be creative, but intentional in using gaze, movement, sound, motion, and controller input as tools the user can employ to take agency.
But remember, above all else, to be sure to design affordances that inform users of the amount of agency available to them, while also teaching them the mechanics of engaging in the available types of agency—especially in the first 60 seconds of the XR experience. Nothing sours an experience like getting stuck on a challenge an hour into the experience because you require users to employ a method of interaction they didn’t even know existed. Firmly establishing the laws of the new world you’re creating is critical to user happiness.
One way of helping users understand their interaction capabilities is by showing them who they are. What form does your user take in your experience? Will the user be human, with familiar capabilities and limitations? A bird that can fly? A wizard with spells at her disposal?
Creating an understanding of who the user is and what role that user is taking on within a given XR experience is one of the easiest ways to afford what users might be able to do. This also kickstarts users’ progress in learning interaction mechanics when they are pursuing the mini-challenges you’ve created at the beginning of the experience for onboarding the user.
While virtual personas help in forming a mental model of users’ interaction capabilities, they are also a great way to set the mood and style of the experience. If the personas are human, what is their sense of fashion? How might this play into the mental model you’re trying to instill in the experience? How do the physical motions your virtual characters take to represent users’ actions fit with the mood you are developing in the scene?
These are just a few considerations. There are many more. More broadly, carefully consider how you’ll virtually represent the user—and whether the virtual representations are in accord with the intended experience, predicted user desires, and the level of agency you intend to offer.
It is important not to dive into scene-level specifics right away. Instead, consider the experience as a whole—based on the overarching experiential intentions of your design. This includes considering how much user agency you want to afford to users—both locally, within particular scenes, as well as globally. Also, consider the types of interactions you want to provide to users, the persona that encapsulates them, and how these influence user mood and the general feel of the experience.
Of course, once you’ve established these experiential building blocks, you must remember to design the first 60 seconds of the initial scene in a way that generates a mental model of the experience to come. By demonstrating affordances and building curiosity, you’ll prevent users from leaving your experience and keep them engaged and moving forward.
These are some of the most fundamental design considerations for creating a new XR experience. In my next column, I’ll cover additional design considerations at a more granular level—such as locomotion and disorientation, spatial-audio grounding, and foveated rendering.