In July 2015, we made an experiment that tested different dialog variants that requested users to change their usage access setting in Android. The dialogs were shown right after login and the users were not allowed to use the app without accepting this setting change. The results were startling. Overall, all variants performed very well as more than 95% of users accepted.
All variants used the basic principle of first explaining the “why,” such as giving airtime or giving the best possible experience, then the “what,” and the “how.”
However, certain variants had higher acceptance rates, as observed below:
Variant 1: 95.9% , Variant 2: 96.3%, Variant 3: 95.1%, Variant 4: 97.2% and Variant 5: 98.1%.
We saw the following patterns:
* We saw an uplift of 0.4% and 2.1% when the message appealed to protecting privacy. This is not surprising as privacy is an important value in India.
* We also saw an uplift of 0.9% and 1.4% when we only showed an allow button vs. 2 buttons that showed options to allow or deny. In other words, the message that only showed a positive action resulted in a higher acceptance rate.
The winning option, variant 5 with a 98% acceptance rate, had both of these options. It also had much more concrete and detailed explanations on why and how to change the setting. See the winning dialog below:
At Jana, we keep running experiments in order to discover better ways to deliver valuable content to our users around the globe. This is one example where the results were a lot more positive than what was anticipated.
Jana keeps hiring! Visit jana.com/careers for more information.