UX Research in the Automotive Industry

Moderated by Susan Shaw, as part of
The UX Researchers’ Guild
This is an abridged version. View the full presentation here.

​In this session, Susan Shaw shares her expertise in UX Research by exploring the most effective methods she has used in the automotive industry.

Types of UX Research

Numerous methods are available and can be used in UX research. As explained on the Nielsen Norman website, the following chart compares “20 methods mapped across 3 dimensions and over time within a typical product-development process.” For more information, visit https://www.nngroup.com/articles/which-ux-research-methods/.

Of these, Susan chose to discuss in this session the following methods which are most common in the automotive industry:

• Usability Benchmarking
• Journal/Diary Studies
• Focus Groups
• Card Sorting
• Interviews
• Usability Testing

Using Usability Benchmarks in UX Research

Usability benchmarking is tightly scripted usability studies using precise and predetermined measures of performance, usually to track usability improvements of a product over time or compare with competitors.

All OEMs (Original Equipment Manufacturers) look at what other OEMs are doing well to improve their own features. An effective way to do this is by creating a form or questionnaire of what you want to know. Then using that as a guide, look at that feature in every vehicle you come across.

The following image shows the icons each OEM uses for the same dashboard feature. They are similar but may not be clear from one vehicle make to another, what the image means.

Other examples of where usability benchmarks are used in the automotive industry include:

• Exploring how many steps users take to turn off Forward Collision Warning systems.
• Understanding how users access the preset button for their favorite radio station, what shape/size the climate control knob is, and whether a driver can reach/adjust these controls without looking directly at them.

Journal/Diary Studies

Diary Studies are used to understand long-term user behavior and experiences, give contextual insights about real-time user behaviors and needs, and help define UX feature requirements. Participants in a journal/diary study use a mechanism (e.g., paper or digital diary, camera, smartphone app) to record and describe aspects of their lives that are relevant to a product or service or simply core to the target audience.

Diary Studies are typically longitudinal over an extended period of time (a few days to a few months) and can be done only for data that participants can easily record. During the defined reporting period, study participants are asked to keep a diary and log specific information about activities you as the researcher, are studying. To help participants remember to fill in their diaries, they may be periodically prompted through a notification received daily or at select times during the day.

(Image from: https://www.nngroup.com/articles/diary-studies/)

In a previous diary study, Susan combined the written log with video recordings. Qualified participants were given an instrumented vehicle (with cameras and other equipment) for 100+ days and asked to log their use daily. In addition to the data from the daily logs, researchers watched the video recordings from the installed cameras to record actual use versus reported use. In this study, she wanted to discover if people used a newly installed alternative technology or defaulted to using what currently existed.

These studies usually take place over an extended period for multiple reasons: (1) to get long-term data and (2) to ensure people were comfortable and past the initial awkwardness and acting naturally. Over time participants tend to “forget” cameras are there and act more naturally.

Focus Groups

Focus groups are an informal technique to assess user needs and feelings both before interface design and long after implementation. In a focus group, 4-15 users come together to discuss issues and concerns about the features of a user interface. The group typically lasts about 1-2 hours and is run by a moderator who maintains the group’s focus.

In a focus group setting, spontaneous reactions and ideas can be more readily observed.  In these groups, you can ask participants to discuss how they perform activities that span many days or weeks: something that is expensive to observe directly. However, researchers can only assess what customers say they do and not the way customers actually operate the product. Since there are often differences between what people say and what they do, direct observation of one user at a time should supplement focus groups.

Problems that can arise with focus groups is when one person monopolizes the conversation, when a participant doesn’t talk, or when someone just goes along with the crowd. The key to getting good data from a focus group is:

• To have a good moderator, one that not only keeps the group on task but also,
• Ensures that opinions and feedback from all group members are being represented.

(Image from: https://www.questionpro.com/blog/focus-group)

Card Sorting

Card sorting is a quantitative or qualitative method that asks users to organize items into groups and assign categories to each group. This method helps create or refine the information architecture of a site by exposing users’ mental models.

Part of making a site easy to use is organizing information so that people find what they’re looking for. Too often, content is structured based on what makes sense to the company, not to the users.  One of the primary ways to figure out an organization scheme that best matches users’ mental model is through card sorting.

Let’s imagine that you’re designing a car-rental site. Your company offers around 60 vehicle models from which customers can choose. How would you organize those vehicles into categories where customers can quickly find their ideal car rental? Your company might use technical terms such as family car, executive car, and full-size luxury car. But your users might not understand the difference these terms. This is where card sorting can help: ask your users to organize vehicles into groups that make sense to them, and then, see what patterns emerge.

As it relates to the automotive industry, Susan has done studies to determine the best locations for controls on a vehicular dashboard. She created three categories: Steering Wheel, Touch Screen, and Center Stack.  Her intent was to discover which controls needed to be duplicated and which did not. Instead of always relying on post-its and paper, she and her team have also put users in the vehicle and given them actual-sized buttons with velcro and asked them to place them where they best saw fit/what made sense to them when driving down the road and needing to make an adjustment.

Interviews Versus Usability Testing

A user interview is a UX research method during which a researcher asks questions about a topic of interest (e.g., use of a system, behaviors, and habits) with the goal of learning about that topic. Unlike focus groups, which involve multiple users simultaneaously, user interviews are one-on-one sessions, although several facilitators may take turns asking questions.

(Images from:https://www.nngroup.com/articles/user-interviews/)

Although you may feel that doing a UX user interview is simple and straightforward, there is more to a good interview than many people realize. You need to know how to dig a little more, make participants feel comfortable and willing to talk and keep participants on track.

Interviews give insights into what users think about a site, application, product, or process. They can point out what site content is memorable, what people feel is important on the site, and what ideas for improvement they may have.

Usability Testing/Clinics

Usability testing participants come into a lab, one-on-one with a researcher, and given a set of scenarios that lead to tasks and usage of specific interest within a product or service. In a usability-testing session, a researcher (called a “facilitator” or a “moderator”) asks a participant to perform tasks, usually using one or more specific user interfaces. While the participant completes each task, the researcher observes the participant’s behavior and listens for feedback. These sessions are best done in person but due to the pandemic, many have had to be done remotely as well.

(Images from: https://www.nngroup.com/articles/usability-testing-101/)

Which UX Study is best

So, now that you know a bit more about the most common research methods in Automotive, which one should you use? No one method is best in any given circumstance. It all depends on where you are in your research, what you are trying to find out, and how much information you want and need.

The following chart is a helpful guide to determine which method will work best at which stage of a research project.

You also need to understand what your budget is before looking at how many participants are involved, or where the testing will take place. Susan shared that when she was doing a study on Lincoln Sedans, she focused on an older population. In contrast, while testing the KIA Soul, she’d go to campuses or large cities such as Los Angeles or New York where she would find younger populations.

While this session focused on identifying methods for the automotive industry, these principles can be applied to any project. Be willing to do the work to understand your project, stakeholders’ expectations, and user involvement to create the most effective and successful study under any situation.

About Susan: Susan Shaw has a BSE, in Systems Engineering from the University of Illinois and an MSE, in Industrial Engineering from the University of Michigan, and is a Certified Professional Ergonomist. She has worked as an ergonomics engineer and consultant in automotive usability.

For seven years Susan was a Senior Human-Machine Interface/UX Engineer at Hyundai Kia America Technical Center, Inc. She is currently the ADAS Customer Experience Research Lead at Ford. Susan is also a Board Member of the UX Researchers’ Guild.