Why context is vital for great Market and UX Research
When it comes to decision making the importance of context must not be underestimated. Without context, it is very difficult to understand the why? It thus explains how good market research has historically relied on both qualitative and quantitative data as a blended approach to aid decision making.
However, in recent years, quantitative data has been to the fore as the primary source of data on which to make decisions. We have data for pretty much everything these days. We have quickly gotten to the point wherein most data-driven organisations are generating huge pools of data and often lack the ability to process it so as to derive value from it. Similarly, as the Nielsen Norman Group claim, “Number fetishism leads usability studies astray by focusing on statistical analyses that are often false, biased, misleading, or overly narrow. Better to emphasize insights and qualitative research.”
The growth of quantitative data has coincided with the IT revolution and as more and more companies undergo digital transformation the volume of data continues to grow.
However, the flip side to this revolution is that we are losing the personal connection with our customers, users, clients and patients. A “digital divide” occurs as in-person relationships decline and we remove empathy, context, nuance and understanding. Take banking as one small example. At one point banking was primarily face-to-face and carried out in the branch, where customers knew the staff and the staff knew them. For many of us, these daily visits to bank branches are now infrequent affairs leading to a distance between the bank staff and customers. Of course one can extract pros and cons from any transformation, and this efficient low touch drive may make more sense from an economics perspective where the bank is concerned, but it also serves to increase the digital divide whereby service providers know less and less about their customers. Looser relationships signify weaker ties and ultimately less brand loyalty.
As increasing numbers of data scientists are employed by these digitally transformed industries a lot of number crunching goes on - but without qualitative research to help make sense of the data, it is easy to be blindsided, and to interpret data erroneously.
When I read reports from other people's research, I usually find that their qualitative study results are more credible and trustworthy than their quantitative results. It's a dangerous mistake to believe that statistical research is somehow more scientific or credible than insight-based observational research. In fact, most statistical research is less credible than qualitative studies. Design research is not like medical science: ethnography is its closest analogy in traditional fields of science.
A good starting point is to take a sceptical approach to any data being presented, as well as to seek access to the primary source so you can interpret the data first-hand yourself. It is all too easy for data to mislead, and some insights can be based on flawed assumptions. Undertaking qualitative research in support of quantitative analysis can help to fill in the blanks, but if this is not possible then ensuring that there are robust discussions regarding the integrity of data is highly recommended.
The following two examples help to illustrate the point:
1. Apples to Apples
Early on in my career as a fledgling Chief Marketing Officer, I was blind-sided by a question from my Chief Executive Officer at one point. He approached me and claimed that his primary investor had told him another portfolio company had a 10% conversion rate on their home page and that he felt a) I did not know ours and b) he felt ours was likely to be a lot lower. I would describe the tone as “accusatory” and I felt I was being criticised for not knowing this key figure “on the spot”.
Having reflected on this afterwards several things became apparent.
Firstly there are no “apples to apples” comparisons without knowing the context. We were working in a B2B SaaS startup, I had had no insight into what this other company did. Maybe they were offering something for free as their main *call to action* on their homepage which was boosting their conversions. Where was their traffic to the homepage coming from? Was it from Google Ads or some other source? The more I thought about it the more it became clear that it was a really bad question, and I should have said so, but for me, it was too late: the seed of doubt had been sown - I wasn’t on top of my game.
Another piece of information that would have informed the debate related to context. We were a small startup - I was the first senior marketing hire and what I had inherited was not pretty. I had a thousand things on my plate, and I was focused on putting the building blocks in place to ensure future growth. That had included an entire website revamp and brand identity refresh, as well as recruitment, which represented fairly significant undertakings for a small team. Again, had I provided the additional context it would have helped assuage his fears. Home page conversion was a lower priority issue at this point of time given we had more pressing issues with larger commercial impact to fix. The big life lesson for me was that context is everything, and one quant data point is meaningless without context.
2. Manipulating Raw Data
Another example that may help illustrate the point further relates to Google Analytics, a powerful analytical platform provided for free by Google and used by most websites around the world.
The data helps you understand the activity on your website in terms of numbers of visitors, where they come from and what they do on the site (as well as much more). However, over the years I have learned that the biggest danger with the data from Google Analytics is how it can lead to bad decision making if it is not subject to *robust interrogation* or if the context is not considered.
Take a report on on-site traffic as one example. When you report on site traffic it helps you assess visitor numbers (or sessions), whether they are growing or not as well as conversion rates i.e. the percentage of visitors that take a desired action. However, raw data can be very dangerous. I will typically apply numerous filters to the raw data to avoid over-reporting. Here are some of the things I strip out:
Visitors from our own office (Internal IP address)
Visitors who come to the site to log in (Existing clients)
Bot traffic
Traffic that stays for less than 1 second
One could go even further, stripping out visitors from countries that your solution is not intended for. Once these datasets are removed the picture will usually be very different, impacting key performance indicators like conversion rates as discussed above. Again the lesson here is that quant on it’s own needs to be interrogated and analysed to add a layer of context to help ensure the insights being derived are credible.
The Importance of Context in market research
‘“Context” is one of those words you will encounter again and again, without anyone offering anything like a useful definition. It is something of a catch-all word usually used to mean “all those things in the situation which are relevant to meaning in some sense, but which I haven’t identified”. Williams NR. How to get a 2:1 in media, communication and cultural studies. London: Sage; 2004
The point from both of these stories is that context is really important when making decisions from quantitative data.
So what should you do?
When it comes to major decisions the voice of the customer, client or patient, must be brought to bear on the decision. This voice will typically come from qualitative market research whereby you get to hear people explain their motivations and insights as to the why?
The challenge with qualitative research is that it can be perceived as being time-consuming and expensive. It is often associated with major research agencies and big brands. It can thus be viewed as an unnecessary expense for those conducting research in more stressed conditions i.e. where resources are more limited. This is a big mistake - unpacking “the why” is key and for that we need insights from the various actors the data is meant to represent.
The Evolution of Qualitative Research
In recent years we have seen the emergence of alternative approaches to in-person research using mobile ethnography [what is mobile ethnography?] as a complementary approach to focus groups and in-depth interviews (IDI’s). Covid-19 meant that in-person interviews or IDI’s and focus groups were no longer conducted due to safety concerns. The business model of running focus groups was also scrutinised as the idea of bringing large numbers into offices in the middle of big cities served to increase the costs of research significantly taking it beyond the means of companies with smaller research budgets.
Mobile ethnography leverages the power of smartphones to enable and empower respondents to participate in market research remotely from the safety of their own offices or homes. This approach offers a cost-effective alternative but also helps researchers to really understand the context. It provides a window into people's daily lives as they record videos from bedrooms, kitchens and visits to the local store. As they move and record we get insights into their daily realities providing rich context for their answers. Smartphones tend to be an extension of the person, and for many people, they are always with them and these portals into their lives help researchers to really understand the why?
The emergence of Mobile Ethnography in Market Research
Mobile ethnography projects tend to be task-based and the use cases are wide and varied from user journey maps to video diary studies to UX research. The following represent some of the elements we believe makes mobile ethnography so powerful.
Show me trumps tell me every time
“In the moment” footage is much stronger than the power of recall
Seeing is better than hearing
A video with audio provides more context than an image
A private window on someone's world is more authentic than what they say in front of a group
Summary
In summary, we believe that “in context” is much better than “out of context” and that the window mobile ethnography provides on the world helps researchers make better decisions. Adding context to research helps ensure that any insights (and likely resultant actions) have the benefit of the blended approach where the quantitative data provides some evidence and the qualitative data explains the why enabling researchers to move forward in confidence.
I’ll leave the last word to Nielsen Norman
“Quantitative studies must be done exactly right in every detail or the numbers will be deceptive. There are so many pitfalls that you're likely to land in one of them and get into trouble. If you rely on numbers without insights, you don't have backup when things go wrong. You'll stumble down the wrong path, because that's where the numbers will lead.”
We get insights from speaking to our users, customers and clients. And when you marry quant data with insight and context you get much closer to the truth you are looking for.
Testimonial: what our customers like about us
Check out what our customers say about us on G2 Crowd