Updated June 28, 2019. While working on various IT projects, I realized that regardless of all these “lean” product development movements, many companies still encounter major issues in terms of product validation. In other words, it’s hard for people to understand if the product they develop is what the market needs. To help you with your validation efforts, I’d like to give you an inside look into the day-by-day Agile product validation process for a live product that we have worked out with one of our clients.
What is product validation?
Let’s start by answering the fundamental question: what is product validation?
There are many approaches to what’s called “product validation.” Try to Google the phrase and you will find that there is more than one way to define it. My short explanation would be that product validation is a process aimed at answering the following question: is my product something that people need? When you build a product, it’s a good practice to verify its value each time you release a new version
One of the most famous frameworks in this field was presented in The Lean Startup by Eric Ries (you can find it on our list of must-read books for CTOs). He suggests following these steps:
1. Build an experiment to test if people need what you are trying to sell them.
2. Measure the result of your experiment.
3. Learn your lesson and then iterate by building your next experiment.
What I like about this approach is that Eric reveals that in order to gain value out of the cycle, you should plan it in the reverse order. First you need to know what you want to learn about your users. Next, you should think about how you will measure it. It's only at the end that you should try to figure out how to build the fastest and cheapest experiment to gain that knowledge.
An inside look at our team’s product validation process
I personally think that product validation is one of the greatest challenges in the whole workflow. In the project I'm currently work on, we encountered the same challenge. However, we decided to structure a process that would support our validation. We don’t call our approach the lean startup framework, although ultimately it does include the 3 aforementioned steps: build, measure, learn.
What does our validation process look like?
1. The pre-development phase: “how to measure?”
a) The Product Owner (PO) adds their idea of validation to the user story.
(Not sure what a PO does? Read our article about Product Owner's responsibilities.) Every time our PO adds a new user story to the product backlog, they have to fill in the “how to measure?” field. This approach helps them analyze, prior to any further discussion, how to recognize if the change we're about to make is something that our users need, and if we have enough tools to allow user story validation. See the snapshot of our user story template below—this is the “planning” part of our framework.
%2520The%2520Product%2520Owner%2520(PO)%2520adds%2520their%2520idea%2520of%2520validation%2520to%2520the%2520user%2520story..png)
b) The validation idea is discussed with other people during the weekly meeting.
The meeting should be attended by the Product Owner, UX designer, Scrum Master, and the client’s representative (optionally).
The way we run this session is very easy: the PO presents their idea for validation while the other participants discuss it and suggest improvements where applicable. This way, the PO receives a wide range of opinions and makes more informed decisions.
2. Development phase
No validation work is done during this stage. This is the “build” part of the lean startup framework.
3. Post-development phase: product validation
After the new product increment is deployed to production, our validation process starts. Some of it happens ad-hoc—our data analysts start producing stats. However, the main point of the process is a weekly meeting during which the Product Owner, UX designer, Scrum Master, and, optionally, the client’s representative, discuss what has been released to our “live” environment and what results it's generated. In other words, we validate whether the new product features had a positive, neutral, or negative impact on end users. For this purpose, we use Google Analytics, a heatmap tool, and other data analysis tools.
After we collect the necessary data and validate new features, we make decisions regarding future development plans. We use a Jira board to facilitate the meeting (see below). We take the items that are in the “To verify” column and try to move them to “Verified,” which means that we have actually validated our feature. These are the the “measure” and the “learn” parts of the lean startup framework.
We iterate these practices and the cycle starts from the beginning every next development sprint. To summarize, the validation workflow is presented on the graph below.
What is an impact map and how do we use it?
Aside from Jira mentioned in the example above, we use one additional tool to visualize product validation status—the impact map.
Impact mapping is startegic planning technique created by Gojko Adzic to help businesses achieve their goals.
(We were quite excited to see Gojko Adzic himself retweet this article soon after it was published.)
The idea is simple. First, you define a business goal, e.g. “Sell our product to a million clients within 3 months.” Gojko suggests thinking about impact mapping as if it were an exercise in navigating a map from point A to point B. In this case, point A would be where we are right now, and point B is the goal we would like to reach.
If we look at the map, there are usually many ways to reach the same goal. The idea is to hit the road and check if all the roads are passable. It might turn out that some are actually closed, or there may be roads that don’t exist just yet.
Using impact mapping terms, you are supposed to define 4 elements:
1. Goal
Decide what would you like to achieve from the business perspective. Maybe you'd like to reach a certain volume of sales in a given period of time, or maybe you'd prefer to focus on the number of new registered users. Whatever your goal is, try to define it as precisely as possible.
2. Actors
Think about various groups of people who can help you achieve the goal. These can be your segments of users, your employees or anyone else.
3. Impacts
At this stage, you need to think about the actions your actors can take to help you achieve the goal.
4. Deliverables
Try to define precise product features that will help actors make an impact.
An example could look like this:
If creating a deliverable drives actors to make an impact that supports our main goal, we validate this deliverable positively. Using that information, we then plan our future work. If a deliverable doesn’t create any positive change, we validate it negatively, which is also a significant part of our learning.
An example of what our impact map looks like
The central part of the impact map (dark blue) presents the main goal we'd set as a team. It can be any sort of a business goal, e.g. 100,000 new users registered in our application in the next 3 months. It should be something precise and easy to measure at the end of the specified period.
On the borders of our impact map, we presented the deliverables (yellow, red, black, and green) we would like to offer to the end users in order to support our main goal, e.g. a free gift for everyone who registers within the specified period. In the Agile environment, deliverables would be called user stories or product backlog items.
The colors of the deliverables carry additional meaning:
- yellow—for all deliverables that were added to the impact map and are in the pre-development stage,
- black—the deliverable is either in the development or post-development stage but prior to the validation stage,
- green—the deliverable was validated positively,
- red—the deliverable was validated negatively.
In our case, the impact map is updated weekly and overseen by the Scrum Master. However, the Product Owner or anyone else familiar with it can make sure the map stays up to date.
Based on the information we have visualized on the impact map, we make data-driven decisions about the future development of the product.
What are the benefits of impact mapping?
Impact mapping has a number of advantages your software house can greatly benefit from. Among others, impact mapping:
- helps maximize customer value and minimize waste by making it easier for teams to focus on the objectives they’re working to reach;
- visualizes the process in an accessible way for everyone involved;
- increases collaboration within the team and unlocks creativity by facilitating the collective creation and testing of assumptions;
- facilitates horizontal working, rather than top-to-bottom;
- helps the team align their work with business objectives;
- makes it easy for team members to spot challenges immediately and respond to them in a timely manner, as well as adapt to changing circumstances;
- keeps the focus on people, their experiences and ideas.
Final thoughts
To recap, here’s a few reasons why impact mapping for Agile product validation is definitely worth your time:
- Creating an impact map helps the team remain focused on the business goals, i.e. the actions that actually create revenue.
- It allows you to see the whole landscape of all possible routes toward your business goal—including faster, easier, cost-effective routes you may have missed.
- Impact mapping opens you up to more creative decisionmaking; you can look at whole sections of the map and be able to see which sections have been effective and which ones you should give more consideration to.
- There's always a different section of the map you can focus on.
- Thinking with the map helps you stay motivated.
Hopefully this post will help you organize your product validation process within your team. Good luck, and let us know how it goes!
If you're looking to learn more about effective software development processes, we have a few articles you might enjoy:
