Michele Ronsen, Founder and CEO of Curiosity Tank, led a recent Remesh webinar on the role of bias in introducing incentives to research respondents. You can watch the full recording here. According to Ronsen, there are at least 180 documented types of biases. Some are captured below, followed by additional insights on bias in research design in our conversation with her described below.
According to Ronsen, in research studies, the more perspective and experience you have, the more positioned you are to shape and identify bias. Self-awareness is the first step in the identification of bias. Then a conversation with one’s colleagues and team around biases assumptions, whether it’s about the topic you are learning about, the stimuli you are testing, the moderator, participants, or even the researchers. Bias is best analyzed in “360 degrees”. Among the top ways to reduce bias, suggest Ronsen, is to involve more colleagues. It is always good to have multiple perspectives involved in the study including a designer, product manager, a more senior member of either the research or product team, sometimes a content strategist and someone from engineering --typically around 4 to 5 people. A pilot for every study is critical and providing feedback on the pilot and the context are important to reducing bias in any research study.
While income levels might have an impact on respondents’ motivation to participate in an incentivized research study, it really is not the central reason why people participate. Moreover, it is respectful and a good gesture to understand the time commitment a study involves, and incentives demonstrate that researchers care about the respondent’s time— especially critical to time-scarce professionals like doctors and attorneys. An incentive could be monetary or non-monetary. Her own clients tend to orbit in the big tech space, where access to food and shelter is not an issue.
With team biases, says Ronsen, teams are much closer to the problem space and frequently think they know the best solutions prior to conducting research however they are often very surprised by participant’s feedback! Being cognizant of such (self-confirmation) biases mixing up the sequence you show concepts to participants, and making sure the moderator is prepared is one of the best ways to mitigate “bias”.With team biases, says Ronsen, teams are much closer to the problem space and frequently think they know the best solutions prior to conducting research however they are often very surprised by participant’s feedback! Being cognizant of such (self-confirmation) biases mixing up the sequence you show concepts to participants, and making sure the moderator is prepared is one of the best ways to mitigate “bias”.
While Ronsen talks about improv being an important skill or tactic to exercise in conducting research, she emphasizes the importance of preparation and structure in every study. A great moderator and user researcher leverage the fundamentals of improv when getting into character. Thinking on your feet and pivoting happens on the job, and that’s where the improv comes into play especially when respondents don’t respond the way you expect. There are opportunities to pivot throughout the study, and getting into character and into the mindset with the confidence to pivot is learned with time and experience, she suggests.
Is there a place for AI in reducing bias in research studies? When talking about AI powered bias detection tools in the marketplace (example BiasCorrect, a Slack plugin app that flags problematic phrases or language before use in the workplace used by the bank of Nova Scotia), she says she hasn’t encountered many. The prevalence of co-parenting apps that mediate language between divorced co-parents, to make sure their communication is respectful, sounds like an interesting application that could or might extend to research design. Co-parenting apps for people who share custody of a child can help you monitor the language and tone you use when communicating. The detector feeds off some key words, tone or even emojis if the communication is via text. Its goal is to help partners communicate in a respectful manner before they regret their interactions. So much has to do with tone, even in any research study, points out Ronsen.
When developing a set of best practices or guidelines that control for bias, Ronsen reminds that it is critical to first identify biases with yourself and your teams. Then lay out those assumptions in order to know how to check them at the door while in session. While there will never be such a thing as bias-free research, we can reduce it. Self-care is another important element for researchers that calls for taking the time before a study to make sure we are recruiting the right people and screening them properly and reviewing the discussion guide to make sure it is fair and balanced so that we are fully present.
The goal with research is about soliciting multiple perspectives to reduce inclinations or biases when you’re in character and when conducting studies. How does one learn to reduce moderation bias? Record all your sessions, go back through your sessions with a checklist, and answer questions very candidly, are this leader's last words of advice: “Record everything. Rehearse everything. Bias comes from so many different sources.”
Further reading: Portigal, NN Group, Curiosity Tank. Sign up for Curiosity Tank's newsletter here.
© Marketing Research and Intelligence Association