3. Designing for Multimodal Interactions
Research has shown that relying too heavily on a single sensory channel for user interaction is not a good thing. Within reason, multimodal interactions are the way to go. So, as much as possible, but without forcing things, plan the interactions within your XR experience to take advantage of multiple sensory channels. Relying only on touch or speech or gestures can be exciting for a short period of time, but the lack of diversity that results from an overdependence on one sensory channel ultimately becomes wearing. A feeling of single-channel sensory overload can be detrimental to the length of user engagement and forestall repeated interactions.
In the days of mouse and keyboard, the user’s input channels were limited, but people recognized and accepted this limitation. However, this is changing. Within multimodal media such as XR, the number of sensory input channels is increasing and now includes options such as speech, gaze, gesture, six degrees of freedom (6-DOF), controller-assisted input, and direct physical interaction using special gloves.
As you design each XR interaction, keep in mind the many options that are available to you. Users aren’t limited to using only one or two sensory channels in real life so in XR, if your goal is immersion, you must try to give users the freedom to interact in many different ways.
Microsoft has done quite a bit of research on how to make the use of holograms in 3D space comfortable for users. Three primary scientific concepts regarding the way our eyes work are relevant to hologram placement:
- Accommodation—Our eyes must adjust their focus when shifting from a focal point that is near to a focal point that is far and vice versa.
- Vergence—When an object moves closer to you, your eyes verge—that is, images move closer to each other—while an object moving further away causes them to diverge.
- Vertical disparity—This occurs when a user focuses on multiple objects that are vertically misaligned.
Each of these three concepts suggests the appropriate hologram placement. If you don’t design holograms properly, by keeping these concepts in mind, this can cause unnecessary visual strain for the user.
A major takeaway from Microsoft’s research on these three concepts, as well as their usability testing of holograms: The ideal zone for hologram placement is within the 2m focal plane. The optimal placement for immersive content is within the 1.25m to 2.5m plane.
Locomotion and Disorientation
One of VR’s coolest superpowers is teleportation. People have always wanted teleportation, but because they’ve never had it in the real world, teleporting is a foreign interaction to them. Therefore, people aren’t biologically capable of handling teleportation all that well. In early VR experiences that included teleportation within 3D spaces, users often felt disoriented and sometimes even nauseous.
4. Reducing Locomotion Disorientation: The Blink Design Pattern
One design pattern that helps address locomotion disorientation is the blink teleportation interaction, which very briefly displays signs of motion upon the user’s teleporting, then almost immediately fades out the canvas, then fades it back in upon the user’s landing at the destination.
This design pattern effectively adheres to the metaphor of motion through a brief visual demonstration that implies, but does not explicitly show the full motion of travel from point A to point B. But by cutting out the vast majority of the motion indicating travel, the blink teleportation interaction reduces VR sickness and helps keep the user oriented.
5. Reducing Locomotion Disorientation: Foveated Rendering
Foveated rendering is not only great for enhancing user focus and precision, it is also helpful in reducing locomotion disorientation. A great deal of the user’s visual discomfort in locomotion is caused by the disparity between the object on which you intend the user to focus and the surrounding objects and environment.
Imagine an XR experience in which the user’s spaceship is in the midst of an action-packed, war scene in space, with missiles from other ships flying by, and space rocks and beautiful shooting stars constantly whirling past. This would be a visually impressive scene, and one that really demonstrates the wonders of XR. However, having too many objects moving around the user can be disorienting.
What if, to advance through this space scene, the user had to interact with various buttons and menus on the spaceship’s control panel? If the user were focusing on a still object such as a control on the spaceship’s control panel, the constant visibility of other objects flying past in space—as well as their related audio information—could be quite distracting and perhaps even disorienting.
One XR design pattern for preventing this disorientation in XR is an interface in which, once the user selects a stationary object, the rest of the scene is blurred, creating a foveated-rendering effect. Plus, you can mute the audio from the blurred objects or significantly reduce its volume to provide better focus.
Of course, implementing this design pattern effectively depends on the interaction at hand, as well as the user’s potential need to interact with multiple objects at once.
While there are certainly many more XR interaction concepts and associated design patterns, focusing on the five essential XR patterns I’ve described in this column will take you a long way toward creating more polished XR experiences.
The advent of XR media is giving UX designers great opportunity to impress, delight, and amplify the capabilities of human beings. With that opportunity comes a lot of responsibility to ensure that the XR experiences you design avoid the many potential experiential pitfalls that can result from human biology. While more research is necessary and our understanding of XR interactions is still developing, UX designers should carefully consider the evolving set of insights about XR design as they pursue its seemingly infinite possibilities.