How to Use Cohort Analysis to Reduce Churn and Make Better Product Decisions

Written by: Ty Magnin Ty Magnin 

Keep tabs on our content. One or two emails a week:

Most startups get it by now—churn is bad. Retention hacking is the new growth hacking—it doesn't matter how many customers your startup acquires if none of them stick around.

There are quite a few helpful tactics that many PMs use to boost their retention, such as improving their user onboarding or increasing accessibility. But to get directly at reducing churn, you'd need to diagnose your product's specific problems and make adjustments.

The good news is that if you're willing to take a dive into the numbers, you can find out exactly why users stop using your app. The key to this is cohort analysis—grouping your users based on their actions to understand what compels them to stick around for the long haul. Then you'll be able to make informed product decisions that will reduce churn and drastically increase revenue.

Enter Cohort Analysis

To find out why your users stop using your app, you have to answer the three W's of user retention:

  • who is and isn't engaging with your app

  • when (and if) they churn

  • why they lose interest

You can only do this by segmenting your users into groups—or cohorts—based on a particular trait. You can segment your users into two types of cohorts:

  • Acquisition cohorts: groups divided based on when they signed up for your app

  • Behavioral cohorts: groups divided based on their behaviors and actions in your app

Acquisition cohorts help you determine the who and the when, but behavioral cohorts enable you to dive into the why.

1. Look at When Users Drop Off

Using acquisition cohorts, you can find out when in the customer lifecycle users tend to drop off. The most common way is to use a chart like the one below.

customer retention cohort analysis graph
[source]

Along the left side is the timeline and the number of customers you acquired at each time interval (the who). Each column represents the amount of time that has elapsed since the users subscribed (the when). Every cell has the percent of the original acquisition number that has been retained at that period in time.

Here are some things to consider when performing your acquisition cohort analysis:

  1. Period (by day, week, or month). Shorter periods for younger companies, longer periods for older companies.

  2. Scope. The larger the scope of the retention period, the more difficult it is to come up with an accurate hypothesis for what's going wrong in the process. Do analyses for each retention period: early (up to 8 days), middle (8-90 days), and late (90+ days).

  3. Expectations. According to Hiten Shah, the retention rate depends on your segment. For a high-velocity, low-priced app, a relatively high churn rate— 10-15%— can be normal. For an app that has a higher barrier to entry, you'll be looking at a much lower churn number—2-3%.

Once you've put together your chart, look at where users drop off at a concerning rate. Is it in day three, once they've been prompted to sync their data or is it in week four, right after they've completed their onboarding? This should give you some indication as to where users are getting tripped up when using your app.

To give you a clearer picture, let's take a fictional data set from a productivity app.

reduce churn cohort analysis

If you look closely, you'll see that the biggest percentage drop-off is right around the two-week mark—the average drop off between day 14 and day 15 is a full three percentage points. That information can help us start making hypotheses about why users are leaving.

2. Find the Sticky Stuff

Once you have a timeframe for when your users are consistently dropping off, you can take stabs at what behaviors caused those drop-offs (the why).

For our productivity app example, we know that we would have to make some adjustments in the early stages of the customer retention period. Behavioral cohorts could help us figure out what's happening around day 15:

  • Users who regularly engage with the checklist feature in the first two weeks

  • Users who regularly use the social features (chat, in-app mail, collaborative workflows)

  • Users who enable push notifications upon first customizing their settings

The correlation between behavior and churn will be more apparent for more specific behaviors. General behaviors, such as “app engagement in the first 30 days,” don't give you much insight into what is keeping users engaged.

Let's put this into practice. Here's the average churn rate for the productivity app, based on our acquisition cohort analysis.

Screen_Shot_2016-06-29_at_11.05.30_AM.png

Here's that same overall churn compared to the churn of users who use one of the core features—the checklist feature.

Screen_Shot_2016-06-29_at_11.08.36_AM.pngBingo! We see that there's a very low churn for the users who engaged with the core feature (the red line), and most of the people who churned did not use this core feature. 


It could be because it was not part of the in-app onboarding, or because the checklist feature is too many clicks away from the home screen. In order to retain more customers, we have to make adjustments to increase engagement with the checklist feature.

As a real-life example, mindfulness meditation app, Calm, was able to 3x their customer retention by pinpointing their stickiest features. They realized that the majority of users who stuck around had engaged with the reminder feature, so they used in-app cues to improve engagement with that core feature.

3. Invert, Combine, and Deduce

Unfortunately, it's not always as easy as just finding one clear link between behavior and retention. As your product and user base grows, it's the combination of behaviors that keeps users engaged with your app.

Think of it this way: your goal is to pinpoint the common behaviors of your most engaged users. Invert that, and you'll come up with the users who aren't sticking around.

All of this can be done in a spreadsheet with some conditional formatting, but that often proves to be extremely time-consuming. Luckily there are tons of tools out there that streamline the process. Tools like Amplitude help you create behavioral cohorts painlessly. You can combine and compare cohorts, quickly testing your hypotheses.

Take a look at how Quizup combined their cohorts to pinpoint their source of churn.

[source]

Quizup saw that they were losing too many users after the first week. They predicted that engagement with their “AddToList” feature would be a behavior that would lead to long-term retention.

Their hypothesis was proven true when they compared their cohorts in Amplitude's Compass tool. Users who engaged with the feature three or more times in their first week were much more likely to stick around.

Quizup inverted the results (looked at users who weren't engaging), and changed their processes accordingly. Based on this data, they made changes to their onboarding, and used push notifications for users who hadn't engaged with the social features (such as “AddToList”).

Iterate and Reiterate

Figuring out how to fix the issue can often be just as difficult as diagnosing it. If you know that user engagement depends heavily on using a core feature, you can't pester your customers with emails and push notifications to force them into engagement. In fact, harassing your customers might increase your churn.

Instead of jumping the gun on big product changes, A/B test modifications on your problem cohorts to get an idea of what works and what doesn't. This way you can make risk-free, but data-backed changes that are guaranteed to reduce churn. And once you've successfully improved your retention based on one behavioral cohort, rinse and repeat.

With so many products available in the marketplace, acquiring customers for a B2B app is no small task. You've put the work into advertising, content marketing, and sales, so don't let those valuable users slip away. All the data to keep them engaged is right at your fingertips.

user onboarding academy

The User Onboarding Academy

Everything you need to know about user onboarding in 7 weekly emails.

Recommended reading