Uncategorized

An Experiment on Push Notification Timing

“It’s hard to over-hype the power of mobile push notifications. For the first time in human history, you can tap almost two billion people on the shoulder.” -Ariel Seidman

At Jana, we are able to tap millions of shoulders around the world through our app mCent. We know that push notifications can be an important channel for communication and driver of re-engagement. I set out to make the mCent push notification experience as smooth and effective as possible.

One of my questions was – what is the best time to send re-engagement push notifications?

As a first step, I made an educated guess by looking at when members use mCent across different times of day. I did some research on push notification timing where I learned that best practices vary widely across apps and audiences. Advice ranged from sending pushes during afternoon hours from 12pm-5pm to adjusting for time zones to prevent waking users at odd hours. There were a lot of possible choices, but no clear answer.

mcent_users_time

This graph is displayed in Boston time (10:30pm Boston time is 8am Indian time).

To answer my question, I set up a simple experiment in Localytics, an analytics and marketing platform that facilitates testing push notifications. I set up the experiment as follows:

  • Messaging – Same message sent to all users (“1+ new offers – try them now!”)
  • Audience – Users in India with English language who were active in the last 30 days, and did not have a session in the last 7 days
  • Timing – Sent at four different times in Indian Standard Time – 8am, 12pm, 4pm, 8pm
  • For each time, the push was sent to 3% of the audience and compared to a control group.

Results from the experiment supported that 8am was the most effective time to send the re-engagement push notification.

Click-through rates at 8am were 3x higher than those at 8pm, and 2x higher than those at 12pm and 4pm. In addition to click-through rates, I also tracked behaviors after engaging with the push notification and compared each to the performance of a control group:

  • Engagement, defined as the average number of sessions per user within 7 days
  • Conversion, defined as the average occurrences of starting an offer in mCent within 7 days
table

This table shows percent changes in engagement and conversion for each push notification relative to the control group, which received a “silent” push notification.

One of our core values at Jana is to “measure it”. When building our products, we often come across questions that have many possible solutions, and we seek out ways to test and validate our hypotheses. In this case, there were many possible times to send a re-engagement push notification, but through a simple experiment we were able to efficiently identify the optimal solution.

Discussion

2 responses to ‘An Experiment on Push Notification Timing

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s