Data-Informed Design: leveraging Quantitative & Qualitative Research

As a UX engineer, I know how crucial it is to grasp what our users need and how they interact with our products. To get an insight into the users and their interaction patterns, I usually rely on qualitative methods like think-aloud studies and interviews. However, these methods can give a narrow subjective view, be
time-consuming, and use up a lot of resources. That's where numbers and data retrieved through quantitative research come in handy. They help me get a broader, more objective understanding of user behavior.

Before I delve into why I strongly believe in the combination of qualitative and quantitative research, let me first explain how I define quantitative research in product design.

What is quantitative research in product design?

For me, quantitative research involves the systematic collection and analysis of numerical data to quantify user behavior and interaction patterns. This process entails tracking metrics such as page views, click-through rates, conversion rates, and time-on-task. To capture these metrics, a wide range of tools can be utilized, with Google Analytics and Mixpanel being among the most popular choices.

One thing I want to emphasize is that quantitative research isn't just about numbers; it's about combining and aggregating these metrics to create a data-informed view of user engagement, behavioral trends, and the overall performance of our designs.

How can quantitative research be used during product design?

Analytics can be used in several ways during product design, but the three tasks I most commonly address with quantitative research are identifying user pain points, testing hypotheses, and understanding user interactions.

Identifying user pain points

Analytics can help identify where users are struggling with the product. For example, high bounce rates on a particular page could indicate that users are having trouble finding what they're looking for. By identifying these pain points, designers can make improvements to these pages to make the product more
user-friendly.

Testing hypotheses

Analytics can be used to test hypotheses about user behavior. For example, a designer might have created two designs and hypothesize that a specific user profile will engage more with one design, while other profiles might prefer the other design. By tracking click-through rates, feature usage and time spent on the page, the designer can determine whether the hypothesis is correct or not.

Understanding user interactions

Analytics can also provide insights into how users are interacting with the product without having a hypothesis beforehand. For example, by tracking how users interact with a new feature, designers can identify optimizations to the new feature or the integration with other aspects of the product.

How to combine quantitative with qualitative research?

However, once you start gathering and analyzing these metrics, you will quickly notice that they are not always straightforward to interpret. For example, if users spend more time on a page than expected, does that mean they are confused, or is the page more interesting to them than anticipated?

Another issue is that these metrics often point out problems, such as users dropping out in a certain step, but don’t provide inspiration on how to solve the problems.

For that reason, I strongly believe in combining quantitative and qualitative research. Qualitative research provides deeper insights and inspiration for solutions, while quantitative research ensures that you don’t miss the broader insights about your user base. Depending on the context and the product's state, you may begin with either quantitative or qualitative research.

However, ultimately, striving to combine them is crucial to gain a more comprehensive understanding of user behavior.

Let's review the three tasks for which I utilize quantitative research and then explore how qualitative research can further enrich the insights we have obtained.

A typical setup of a think-aloud study

Identifying user pain points

In our example, we observed a high bounce rate on a specific website page. You could begin by analyzing the analytics data to identify the issue. You might notice that users are spending very little time on that page before bouncing. Based on this data, you could hypothesize that users are struggling to find what they're looking for on the page.

To validate this hypothesis, you could conduct user studies, such as usability tests or interviews. During the study, you could ask users to perform tasks related to the page and observe their behavior. You might discover that users are not struggling with the page itself but, rather, are confused by the navigation on the preceding page, leading to users ending up on the wrong page.

With this information, you could enhance the navigation to guide users to the pages they are seeking and perhaps include breadcrumbs on the page itself to help them navigate more effectively to the correct pages.

Testing hypothesis

Suppose you have a hypothesis that novice users are more likely to engage with products if you show them visually pleasing product cards while expert users might engage more with a table view with more information and product details.

Once you conducted the A/B test, you might find out that novice users did not engage more with the cards, but neither did they like the table view. To get a better understanding of this behavior, you could conduct interviews to understand why novice users were not engaging with both views. You might find out that the card-based view did lack some essential information for them while the table view was too overwhelming.

Using this information, you could improve either the table or the card-based view to fit the information needs of novice users.

Understanding user interactions

Suppose you want to understand how users are interacting with a new feature on your website. You might start by tracking analytics data related to the feature, such as click-through rates, time spent on the feature, and conversion rates.

It's highly likely that you will observe unexpected patterns. To gain a deeper understanding of this user behavior, you could conduct additional think-aloud studies. In these studies, users can clarify their reasoning behind the unexpected interaction patterns.

You might discover that users are having difficulties with a specific aspect of the feature, such as a confusing interface or unclear instructions. In this case, you could make improvements to the interface or the instructions.

Conclusion

In conclusion, combining quantitative research with qualitative research is a powerful approach to product design because they both provide unique perspectives on the behavior and needs of your users. By using these methods together in an iterative process, designers can develop a more comprehensive understanding of users and make data-informed decisions to enhance the product.

So the next time you're designing a product, consider not only conducting qualitative research but also incorporating quantitative research into your process. This way, you can gain a more complete picture of your users.