A common misconception is that the bulk of the work on your digital product is done by launch day.
Nothing could be further from the truth. In fact, that’s when the real work begins.
Once you launch your product, you start getting tons of feedback, both positive and negative. It’s very exciting, since actual people are finally using this thing you’ve worked so hard to build, but it also means there’s a mountain of bugs to fix and changes to implement.
So how do you go about it? How do you choose the features to prioritize? Or decide what metrics to track and how?
Don’t worry. We’ll tell you all about it.
Read on to learn the ins and outs of UX analytics, and how to leverage them for your product design.
We’ll tell you what UX analytics is and why it matters, how to best do your research, what your process should be, and the tools you can use.
At the end, we’ll also give you an example of how UX analytics has helped us with one of our own projects.
What is UX analytics? Why is UX analytics important?
UX analytics is a crucial, yet often overlooked element of product development.
Think of UX analytics as an ongoing process that validates your current development efforts and helps you make a decision to pivotif necessary.
This data helps you focus on your business goals to drive analytics measurement plans, instead of tracking superficial metrics.
Having said that, it’s also important to identify the core goal of your design for the entire duration of product development. It’s the only way to meaningfully measure the design throughout its life cycle.
Qualitative and quantitative research
There are two main approaches to research: qualitative and quantitative.
Let’s talk about each of them for a second.
1. What is qualitative research?
Qualitative data offers a direct assessment of the usability of a given product’s design.
Designers watch participants struggle with specific UI elements and infer which aspects of the design are problematic and which work well.
Session recordings are an example of qualitative research.
2. What is quantitative research?
Quantitative data offers an indirect assessment of a given design system’s usability—in other words, we’re talking numbers.
These numbers can either:
- be based on the users’ performance of a given task (e.g. task completion times, success rates, number of errors);
- reflect the participants’ perception of the design’s usability (e.g. satisfaction ratings).
A great example of quantitative research are Google Analytics.
3. Which is better: qualitative or quantitative research?
The question may seem intuitive, but it is, in fact, incorrect.
The answer is: neither is better. Because that’s not what this is about.
What you should do is combine qualitative and quantitative data in order to fully understand the reasons and motivations behind the trends and behavioral patterns you’re observing.
This combination of the qualitative and quantitative approaches allows you to evaluate the performance of your UX design and your whole application in a holistic and coherent way.
4. How can you combine qualitative research and quantitative research?
Qualitative and quantitative research can be used together to further drive the direction of product development.
A good way to combine them is to use these two sets of strategic questions:
- What happened? → Why did it happen?
- Is this unusual? → Is it happening to more users?
Note that each of the sets assumes a different relationship between the two types of research.
What happened? → Why did it happen?
This type of research represents a move from quantitative research to qualitative research.
Qualitative research can help you discover variables and explain why certain things happen after you discover them through statistical analysis.
Is this unusual? → Is it happening to more users?
Conversely, the second type of research is an example of moving from qualitative research to quantitative research.
Doing so allows you to discover individual trends and/or preferences, then support these claims using larger samples.
Leveraging UX analytics
1. How it’s usually done
What’s the most common scenario of using UX analytics? Installing the Google Analytics Tracking Code as-is.
However, there are several problems with going about it this way:
- It gives you only quantitative data, telling you the “what” without the “why”;
- Google Analytics needs to be set up correctly in order to give valuable insights (events, segments, etc.);
- Managing it usually requires a full-time Analyst position to make sense of all of the data;
- It’s like looking for a needle in a haystack—tracking the wrong metric will be fruitless and eventually prove meaningless.
2. How it should be done
Fortunately, there’s a better way to do this.
The rule of thumb here should always be: define your goals first, then decide on the metrics.
If you don’t define the goal clearly, it will be impossible to determine the best method of tracking the performance or user experience.
Here are more specific steps you can take:
a) Define the KPIs
Develop a strategy by setting up analytical goals.
b) Select the toolkit
Choose the right set of tools for a particular context—desktop vs. mobile users, new vs. existing users, etc.
c) Create the scenarios
Map the scenarios for measuring results based on existing user flows.
d) Develop team cooperation
Create an implementation brief for the development team, then check the feasibility.
e) Process the data
Draw conclusions and formulate new test hypotheses for further exploration.
UX analytics tools overview
Up-to-date knowledge of various analytical tools ensures that the team's time and resources are used in the optimal way.
Useful functionalities
Tools
QUANTITATIVE
Heatmaps
Understand what the users want, care about, and do on your site by visually representing their clicks, taps, and scrolling behavior.
FullStory, Hotjar, Inspectlet, Mousestats, UXCam, AppSee, Smartlook
Conversion rate funnels
Find the biggest opportunities for improvement and testing by identifying on which page and at which step most visitors are leaving your site.
Google Analytics, Hotjar, Heap, AppSee, Smartlook
Form analysis
Improve online form completion rates by discovering when and why your visitors abandon your forms.
Hotjar, Inspectlet, Mousestats
Feedback pools
Understand what your web and mobile visitors want and what's preventing them from achieving it by targeting time- or event-based questions.
Surveys
Collect deeper insights from your customers as soon as they convert, or insights from visitors before they leave, using responsive surveys that work on any device.
A/B testing
It lets you compare the performance of two versions of a page by showing half your visitors one version of your site, and the other half the second.
Google Optimize, Inspectlet, Firebase
User flow / Behavior reports
This feature gives you a big picture of how users are navigating within your app. With this tool, you’ll quickly understand where the majority of your users are making their way between your different screens.
Google Analytics, AppSee, Smartlook
Basic numerical data
Traffic analysis, user segmentation, and page view analysis are all basic sources of numerical data for your audience.
Google Analytics, Heap, UXCam, AppSee, Firebase
QUALITATIVE
User recordings
Identify usability issues by watching recordings of real visitors on your site as they click, tap, move their cursor and navigate across pages.
FullStory, Hotjar, Inspectlet, Mousestats, UXCam, AppSee, Smartlook
Open-ended feedback
Give your visitors an easy way to leave instant written feedback on your website or app. See what people love and hate, identify issues, and find opportunities for growth.
Research implementation: examples from a project
Launching a New Feature
Addressing Feedback
Approach
Quantitative Research → Qualitative Research
What happened? → Why did it happen?
Qualitative Research → Quantitative Research
This is unusual? → Is it happening to more users?
Tools
Google Analytics → FullStory
Mousestats → AppSee
Primary Trigger
A statistical analysis (conversion funnel) showed us that users are abandoning the Apple Pay payment option after an initial decision of using this payment method.
An open-ended feedback insight within the app suggested that it is not entirely clear how to proceed with a designed primary user flow.
Deeper Analysis
Video recording of this particular screen revealed a bugged expansion panel that should contain a summary of the payment option description.
User flow analysis revealed that 84% of the active users from the previous month were constantly moving back and forth between the two steps of an action; they were stuck.
Problem Identified
Users were not able to obtain a crucial information about this payment method and therefore decided to move forward with more familiar methods.
Users were confused by the coloring of the primary buttons. Even though they aligned with the brand’s Corporate Identity, they were indicating a danger or warning (red color) based on color psychology.
Outcome
Removing a simple bug increased the adaptation of this method by 900% in the upcoming month.
Changing the color and slightly tweaking the labels within the buttons decreased the back and forth movement by 54%.
What other questions or opportunity areas can be supported by the right analytics measurement plan?
The potential of UX analytics does not end there. Here are a few more questions that you can answer by implementing some of the tools and methods above:
- How to set up for success?
For a new and recently launched product - How to break up big challenge areas into small actionable tasks?
For mature product evolution and roadmap planning - How to know what to do first?
For data-driven design and development efforts
If you’d like us to dig deeper into any of those, let us know in the comments!
Final thoughts
In order to achieve meaningful measurement of UX analytics, you shouldn’t focus on goals that are easy to track, but rather goals that are:
- high-level,
- UX-oriented,
- business-focused.
Should you fail to consider these deeper goals when selecting your metrics and reporting on your data, you risk overlooking crucial statistics that are imperative to effective Product Design.
Thanks for reading our article. If you found this helpful, you should also:
- read Software Product Design: What Is the ROI of Good UX?
- read Application Form Design: Best Practices and Key Elements
- take a look at our Behance profile
And if you're looking for excellent design and analytics for your product, why not hire our Product Design team for the task?
