When it comes to research methods: quantitative and qualitative methods often get pitted against one another: teams feel they have to consider one OR the other to answer a business question. But in many cases, the answer might actually be BOTH instead of either/or: sometimes a mixed methods approach is going to be the stronger choice.
We sat down with Nurca Yener-Bozkurt, Senior Quant Manager at Conifer Research to better understand the synergies that arise from bringing quantitative and qualitative research streams together. With 20+ years of experience in CPG Insights, and now leading quantitative research at an ethnographic-minded research consultancy, her days are spent bringing the best of these two worlds together.
“At the end of the day, all research is about answering the fundamentals of who, what, when, where, why and how: It always comes down to some combination of those things,” said Yener-Bozkurt. “Having a sense of how much your team knows - or needs to know - about each of these areas ultimately is what helps to inform the recommended research methodology (or which mix of methodologies) should be used.
Taking a mixed methods approach allows us to build towards more meaningful insights by using the superpowers of both techniques together. Yet qualitative and quantitative research often requires two completely different skill sets and mindsets. The right integration of these two things is both an art and a science.
But first, yes: one method can work. Both quant and qual can stand on their own two feet in many research scenarios. Research projects don’t exist in a vacuum: a team or business may already have an immense wealth of knowledge internally, but is missing the in-context details that can help them make sense of the data. In these scenarios, teams may find that they need grounded context and momentum to move forward with their next step through qualitative in-depth interviews or ethnographic immersion to inspire ideation, so a standard single-track program may be just the thing to ensure that teams meet their agile project timeline and business goals.
The unfortunate reality is that a lot of the valuable data and information within companies is very siloed. It can be hard to access or sometimes just plain hard to make sense of, and more often than not is very narrowly defined to a specific area of inquiry. Questions might have been asked in a different way, collected at a different time, or measuring an entirely different context, making the collection of fresh data essential to answering the questions at hand with confidence.
Like chocolate and peanut butter, some things are undeniably better together. Most research projects benefit from using both in a project or sequence of research phases. If a team needs to evaluate a large number of groundbreaking concepts and figure out which ones to move forward with, how to move them forward, and how people interpret them or imagine their future value you’re going to need multiple touch points of research over a period of time, involving both qualitative and quantitative methodologies in a thorough and iterative approach that generates clear direction.
Quantitative methodologies are meant to give you a clear picture of hard data points. The what, how many, who, where and how much are common areas of inquiry. This information can be powerful in the hands of people who know how to use it. Quant data can help you measure, size a market, benchmark against competitors, prioritize products, fine tune a business model or even build prospective business cases for new products. Robust quantitative data reduces uncertainty and increases confidence in "go/no-go" decisions, and even allows teams to run scenarios about how different versions of the same product or service, with varying combinations of features, will perform in-market.
The downside from quant on its own is in what you don’t get: the contextual background and story richness that qualitative research and ethnographic techniques can provide. Quant data, for its immense power, also has big limitations. Quantitative data is not as evergreen as qualitative data: the usefulness of the data is constrained by the dynamics of how a specific question was asked in the original survey - a nuance that sometimes only the survey-author might be fully in-tune with. And as valuable as quant data is, sometimes teams end up being so removed from the intricacies of how it was structured that they end up using it in ways that it was not intended. “With qualitative research, our motivation is usually ‘open-ended’ from the get-go,” said Yener-Bozkurt. “But with quantitative techniques, the questioning lines are by nature more intentional in terms of measuring something specific … with every question, you have to be clear on what it is you’re trying to measure. As you move forward, you must also take care of how you interpret and use that data, so that it does not become manipulated, distorted or misrepresented down the road."
With qualitative methods, we are exploring a space using principles from the social sciences and design research. Our attention to nuances is focused on the responses and the users themselves: it requires an inductive curiosity, rapport and relationship-building, open-mindedness, listening and observing closely of the responses themselves. There is also a certain level of comfort with ambiguity that is essential for qualitative approaches: while we always have specific product or business goals, our starting point is usually simple provocations to start discussions on topics and explore how and where someone interprets it. This allows us to see how someone’s mental models, value system, decision-making, emotions, behaviors, needs and pain points come out organically and in-context: all things that are difficult to measure authentically in a survey.
Without qualitative research, you might not even know the right questions to ask on your survey instrument. “If most of your respondents are selecting ‘other’ on your most important survey question, you know you have a problem that only qualitative methods can solve,” said Yener-Bozkurt. Qualitative research can also help explain and demystify variability, tensions, and conflicts in quantitative data. It's not uncommon to see data where, at first glance, it seems impossible for both things to be true, but by better understanding how a respondent interprets a question (filtering through their world views and value systems) we can better discern what these differences are actually telling us.
Qualitative research is rich, rigorous and can be bottomless in its depth - but a limitation can appear when you have to scale the insights from your smaller, focused sample or selected markets up to a broader, and even global, audience. It is here where quantitative research can complement and add immense value, providing statistical significance and confidence to fully formed insights to help teams better understand how these directives or needs play out on a broader scale.
So when are these methods better when used together? In many cases, you need both the scientist and the artist. Quant is great for narrowing lists of choices down, and when certain choices rise to the top (or even bottom), we can drill down on them using qualitative techniques to gain a better understanding of the ‘why’ and to bring specific areas, key tensions, or complex dynamics to light.
“On a recent study, when the first round of MaxDiff results came back. Ben Jacobson, our Co-Founder, looked at the data and pointed out straight away that the top five concepts from the MaxDiff results were not the five that the team chose to move forward with in the subsequent qualitative phase. His observation was important - why go through the elaborate ranking analytics if you aren’t going to choose the top 5? It is because you have to triangulate the quant results with the qualitative findings to make the most informed choices.” said Yener-Bozkurt. “In that specific example, we chose a few concepts from the top 5, and a few from the middle and bottom ranks so that we could better understand why certain things were performing the way that they did. This helped us strengthen all the concepts through the qualitative concept testing we did next.”
If you’re thinking about whether a mixed method approach is right for you, here are a few unbeatable Quant/Qual pairings that are always better together:
| Quant Especially Appropriate for: | Qual Especially Appropriate for: | |
| Foundational market knowledge; measuring your brand’s health through the brand funnel and comparing that to your competitors | + | Brand health and equity deep dives: to understand the whys behind the strong and weak spots: why is your brand heavily considered, but not purchased; why is trial rate so strong, but not loyalty? |
| Segmenting your customers based on their distinct behavior and psychographics, then sizing and prioritizing your segments to support your strategy | + | Bringing segments to life or prioritized segments to better align marketing, communication and product development efforts on their needs |
| Focusing your limited resources for your innovation pipeline; narrowing ideas with the highest potential and basing decision to move to the next phase on sound, robust data | + | Filling an innovation pipeline, from insights and inspiration to moving from idea to concept, to prototype, to ultimately a product or service in market |
| Optimizing the design and pricing of your new product or service by identifying the right mix of attributes or features and price; measuring the impact on your estimated market share of altering that mix, or adding line extensions | + | Defining opportunity spaces and design criteria through consumer needs, core drivers, pain points and trade-offs they are willing to make to determine the right attributes and levels to focus your product development |
If it is so great, why doesn’t everyone do this? Sometimes what happens is that it's hard to find teams that are really excellent at both quantitative and qualitative methods. This can mean that the workstreams end up getting separated or teams are forced to choose one over another, resulting in a loss of synergy between the two work streams. “Of course, I’m a little biased, but that's the beauty of having one shop like Conifer, that has a multi-disciplinary team with experts in both approaches. We understand design, stories, business, culture as well as hard data and complex analytics, because all of those things are stronger together. With collaboration and this high-level of expertise in real-time, in the moment iteration and richer insights become possible.” said Yener-Bozkurt.
Two Sides of the Same Coin
In a way, quant and qual are like two sides of the same coin – with "hard data" on one side, and "rich context" on the other. Making sure decisions are based on solid, statistically significant data is important with today’s expensive business decisions, which is why research design - and considering both sides of the coin - is so important. “Our foundation in ethnographic research, which is heavily reliant on context, observation, understanding people's cultures, having open dialogues, and exploring things like emotion and behavior, have made us good at looking for connections and synergies,” said Yener-Bozkurt.
Better Together
In the simplest of terms, qualitative should be designed to embrace ambiguity, while quantitative brings structure. One cannot do both. But they certainly can be designed to be complementary and to “feed” one into the other so that in the end, as the old adage goes: The whole is greater than the sum of its parts.