In tech, it’s important to speak to users. The information you learn from qualitative data can be very different and unexpected compared to quantitative insights. However, people are not machines! They have biases and behaviors that can cause the information you gather to be incomplete or even downright incorrect. At Jana, it’s even harder. Because our users are generally not in the US, talking to them often means approaching strangers in foreign countries and casually striking up a conversation about technology and mobile. So – how do you do this without biasing your results?
- Don’t assume they speak English. This may seem obvious, but one thing I’ve learned in India is that people are sometimes able to read English without being verbally fluent. For example, I spoke to one person whose entire phone was in English and who even chose English over Hindi when registering for mCent. However, he wasn’t able to fully communicate in English his perceptions about the app or mobile in general. In these cases, just watch! Even if you can’t get them to say what they think, there are still things to learn from watching how they use the app.
- People want to make you happy. If they know you work for the app you are testing, it’s common for them to simply tell you it’s wonderful. While a nice compliment, this is not helpful. One way to circumvent this is to be very clear about wanting real feedback. Another way is to not tell them that you work for the app. Tell them you are doing a research project, or some other reason, and they will be more likely to say what they really think.
- People are good at testing – but they are better at comparing. If you just show them one app, it’s easy for people to think that the features work well, because they have nothing to compare it to. However, if you have a second app to test (perhaps a competitor!), you can compare specific features and capabilities to get more actionable information.
Qualitative information is hard to gather. Making sure that your data is valid will help each conversation count towards making your product better.