Avoiding Technology Pitfalls
“I want to share just a couple of points to focus on when you’re running remote usability-testing sessions,” answers Michael. “A couple of key factors have helped our teams in initially setting up test sessions and ensuring a smooth process for participants. To prepare your team to encounter technology issues, begin with the mindset that the technology running your remote concept test, usability test, or interview is probably going to fail at some point. This way, you’ll come prepared and have backup plans in place for any given situation that arises. One way to help ensure smooth sessions for your participants is to run connectivity tests a few days before the actual sessions.
“Let’s consider some examples of technology-alignment failures that I’ve experienced during remote research sessions, which could involve both your technology and that of your participants. What if the application you’re using to communicate with the participant fails? Or the recording of a session—assuming the participant permits recording—does not catch the sound or video or fails altogether. These are things you can control, so you should have a backup plan at the ready. WebEx does a great job of recording sessions and organizing its video files and delivers a transcript after the session as well. But I would still suggest having QuickTime running in the background to provide a backup recording.
“Firewalls can cause some technologies not to work together and ultimately destroy a session with what was a willing participant. Before each remote session, run a connectivity test with the participant. This should not take any longer than ten to fifteen minutes. Ask participants to conduct their connectivity test in the same place in which they’ll be participating in the actual session, using the same machine whenever possible. This can help you gauge the quality of their Internet connection, as well as identify any firewall issues. Ensure that you and the participant can hear each other, the applications are syncing, and anything else you might need to cover from a technology perspective for a given session. In my experience, participants are happy to go through this process because it shows we care about their time, appreciate their expertise, and want to ensure a smooth session.
Communicating Changes in Your Research Techniques to Stakeholders
“When you’re preparing your research plan, take into account that you’ll probably have stakeholders from the business or product management who want to join your remote sessions,” continues Michael. “This can be a great step upward in your company’s overall UX maturity level. But how can you ensure that the sessions stay on track? The solution: always have a plan in place for educating stakeholders about how to behave when observing a research session. Let them know that interrupting a call with additional commentary could derail the session. Having people other than the participant and the moderator talking could potentially confuse the participant and throw off the flow of the conversation. Our UX team touches base with any stakeholders who might join the sessions to explain to them clearly what activities the sessions comprise, the role of the moderator, your overall research goals, and of course, what others in the session should and should not do.”
Great Online Resources
Caroline recommends some online resources that provide really useful information, as follows:
“Although the latter post is a few months old and, for now, we’re less locked down than we were, I think it’s still relevant. The post covers topics that many of us must continue to think about—such as taking care of your own health and your colleagues’ health, as well as respecting the challenges that your participants might be facing.”