LESSON 3

Brainstorm

We’ve covered the first two stages of the EMBED user onboarding framework in this academy—Establish and Map. In this lesson, we’ll cover the B—brainstorming.
First, we’ll explore how to use your analytics tool to inform your onboarding tests. Then, we’ll explore additional data points you’ll need to get the full understanding of what’s happening with your current user onboarding.

Last, we’ll brainstorm some potential solutions and tests for you to implement next week. Let’s get started!

Identify drop offs with your analytics tool

Your analytics tool is a treasure trove of information and inspiration—the trick is to know where to look and what to look for.

If you didn’t have an analytics tool before the last lesson, it might take a bit longer to populate with data—anywhere from a few days to weeks before you can derive meaningful insights from your funnel. Be patient, especially if you have a smaller user base. Sometimes early data gives false indications of where to look.  

Once you have enough meaningful data, start looking for steps in your user onboarding journey where you can see a large number of users drop off or important tasks that have high non-completion rates. The steps with levels of drop off that surprise you most might be the right place to start to focus on.
“When analyzing your user onboarding, focus on drop offs that surprise you.”
Also, try experimenting with cohorts and segments. Try segmenting different user properties, and look for any differences in the conversion rate. The beauty of analytics is that it shows you behaviors you’d never expect, so don’t be afraid to play around with different cohorts and dig deeper into your user experience.

The end goal of cohorts is to figure out the kinds of things successful users are more likely to do than the average signups. But if nothing else, this exercise will show you where you are losing users.

Figure out why the drop-offs are occuring

A quick refresher! There are two types of data: qualitative and quantitative. The best teams use both types of data to create a continuous loop of product improvement.
Quantitative data comes from quantities of information. You can use this type of data to figure out trends and run statistical models. Quantitative data helps you focus attention on the biggest problems and opportunities you have.

The issue with quantitative data lies in its very nature. Because it’s expressed in numbers, it gives you the “how much” or “how many”, but it doesn’t give you the “why”. For that, we turn to qualitative data.
“Quantitative data only gives you the ‘how much’ but not the ‘why’.”
Qualitative data is about qualities. It’s harder—if not impossible—to measure. You can use qualitative data to "zoom in" to the level of the user to clarify your understanding of the problem.

It’s a common trap to only rely on one of these types of data, but think of them as two sides of the same coin. Quantitative data will highlight issues in your product. Qualitative data will show you precisely what the why the user is having those issues and might even point to an easy solution.

You can find quantitative feedback using tools like:

  • Google Analytics
  • Amplitude
  • Mixpanel

You can find qualitative feedback by:

  • Watching user sessions (Using a product like Fullstory)
  • Holding 1:1 user testing
  • Interviewing users
  • Surveys—in-app or via email
Now that you’ve identified where the drop-offs are occurring, dig deeper with whichever type of qualitative feedback works for you—or try a combination of a few. The good news is you’ll need less volume of qualitative feedback to find a pattern.

Brainstorm potential solutions and tests

Once you’ve identified a point of drop-off and come up with a few hypotheses, it’s time to begin brainstorming your onboarding changes!

As you brainstorm, keep in mind a few psychological principles that will help you make decisions in what you build:
  • Goal Gradient Effect explains the reason why we’re more likely to complete things that we’ve already started. Try giving your users early wins in your onboarding, like showing a checklist with the first two things already completed.
  • The Choice Paradox corrects the falsehood that more is better. In fact, people are more likely to make a decision (and feel better about it) when offered less choice. Make sure you’re not overwhelming your user with too many options in your user onboarding.
  • The Principle of Commitment and Consistency shows that humans like to make consistent decisions—and they’ll continue to do so especially if the initial commitment is small. Make sure you’re not asking for too much up front in the onboarding process from your users. Ask for as little as possible, knowing that down the road your users will be more likely to continue the commitment.
  • Gamification is the idea that you can apply elements of games (competition, awarding points, etc) to non-game elements to increase the likelihood of engagement. A word of caution with gamification: While it’s a good thing to consider, it’s not a silver bullet. If the gamification comes at the expense the user’s progress, it’s best to leave it out.
  • The Zeigarnik Effect states that humans focus more on uncompleted tasks than completed ones. This is huge for user onboarding! Simply framing to-do items as uncompleted tasks can be a huge win.
  • Familiarity bias is where we favor familiar places, people, or things rather than novel ones. In this case, it might be helpful to use common UI patterns in your user onboarding.

Lesson 3 wrap-up

As you brainstorm your user onboarding tests, keep in mind that they are just that—tests. Try some wacky ideas. Get creative and have fun!

But also keep in mind that each onboarding change you implement should benefit your users—not just your business. To keep on track, ask yourself: Why would my users want to complete this bit of product onboarding? What will it help them do? Will it be faster for them?

Homework for lesson 3:

  • Explore using cohorts in your analytics software to find areas for improvement
  • Search out qualitative feedback to help inform your quantitative data in order to see the big picture
  • Decide where and what you want to change in your user onboarding