Testing Ideas: The Ultimate Validation Experiments Library
Everything the Product Manager needs to validate hypotheses about value, usability, viability, and feasibility.
Hey, Paweł here. Welcome to The Product Compass!
Every Saturday, I share actionable tips and resources for PMs.
Here’s what you might have recently missed:
Premium: Product-Led Growth 101
Testing Ideas: The Ultimate Validation Experiments Library
Many popular resources about experiments in product management are incomplete or focus on just one type of experiment.
Others, while valuable, are a bit confusing.
I believe that:
Value and usability are two separate risks.
Research activities, like exploratory interviews, are not a validation tool.
You don't build products just to test the feasibility.
The cost and evidence of experiments are highly contextual.
So, in today’s issue:
Validation Experiments
Market Engagement
Minimal Products
🔒 Feasibility Prototypes
🔒 User Prototypes
🔒 Experiments in Production
🔒 Recommended Tools to Use When Experimenting
1. Validation Experiments
First, while the scientific definition of experimenting includes exploration, using this definition in the context of product management might be confusing.
Whenever I talk about experiments, I mean experiments used to validate hypotheses (about value, usability, feasibility, or viability):
According to the scientific definition, you could call an experiment:
Every interview (customers, stakeholders, partners).
Analyzing data from product analytics.
Every market research activity.
For practical purposes, I call them “Research Activities.”
2. Market Engagement
Experiments in this group focus on validating how the market will engage with your product idea. My favorite approach is the XYZ Hypothesis (expressed as “at least X% of Y will do Z”).
That way, you can test:
Value proposition
Viability (including pricing)
Messaging
All experiments in this group can be considered MVP Prototypes and Pretotypes.
2.1 Email Campaign
Send a targeted email to potential customers. This allows you to test your value proposition and messaging.
Tips:
Include a call-to-action, such as a survey or sign-up form.
You can use tools like Mailchimp, MailerLite, GetResponse, or HubSpot.
Email Campaign is typically combined with a Landing Page.
2.2 Paid Ads Campaign
Create a paid ads campaign targeted to a specific audience. This allows you to test not only your value proposition and messaging but also cost-per-click (CPC).
Tips:
Include a call-to-action, such as a survey or sign-up form.
You can utilize tools like Google Ads, LinkedIn Ads, or Meta Ads.
Email Campaign is typically combined with a Landing Page.
2.3 Social Media Campaign
Publish a social media post. This allows you to test your value proposition and messaging.
Tips:
Include a call-to-action, such as a survey or sign-up form.
Consider working with influencers to reach a broader audience.
Social Media Campaign is typically combined with a Landing Page.
2.4 Landing Page
Create a simple webpage that communicates the value proposition, capabilities, and benefits of your product idea.
By driving traffic to a landing page, you can gather leads, collect feedback, and test willingness to pay.
Tips:
Include a call-to-action, such as a sign-up form, and limit other navigation options.
The user should demonstrate interest in the product by paying for it in some form (see Skin-in-the-Game points)
You need traffic, so a Landing Page is typically combined with Paid Ads, Email, or Social Media campaigns.
While a landing page might be a Fake Door, it doesn't necessarily have to be. This depends on whether you suggest the product is currently available.
2.5 Fake Door
Test the interest in a non-existing product by suggesting it exists and is available. This allows you to test your value proposition, messaging, and pricing.
Tips:
Apologize or offer something else for free if someone orders it.
A Fake Door might be a Landing Page, but it doesn't have to be. For example, you might add a non-existing product to the cart in a restaurant.
Also known as “Mock Sale.”
More information: pretotyping techniques.
2.6 Facade
Test the interest in an existing but not yet broadly available/scalable product or service by creating artifacts that suggest greater availability (or scale).
Tips:
Applicable typically to physical products.
More information: pretotyping techniques.
Tags: Physical products
2.7 Pinocchio
Create a non-operational version of your product and use your imagination to pretend it works to see if and how you would use it.
Tips:
It tests only your own experience. It might be helpful at a very early stage.
Applicable typically to physical products.
Also known as “Pretend to Own.”
More information: pretotyping techniques.
Tags: Physical products
2.8 Explainer Video
Create a video to demonstrate how your product will work. Explainer videos are typically short online videos. The goal is to create something that conveys the core value proposition and can be shared to gather feedback.
Tips:
Video Explainer is typically placed on a landing page but might also be shared on social media.
Also known as “YouTube” (pretotyping techniques).
Tags: Physical products
2.9 Pre-Order
Offer your product for pre-order before it’s ready. Communicate your value proposition, problems you are solving, and differentiators, and target an offer to your potential customers.
Tips:
In this type of experiment, you collect money. This not only validates the demand but also might provide funds to support development.
2.10 Letter of Intent
Ask the customer to sign a formal note indicating their interest in purchasing a product in the future.
Tips:
The Letter of Intent is not legally binding but, arguably, involves a lot of “Skin-in-the-Game points.” You don't sign a letter of intent with just anyone.
It’s a great way to validate the willingness to buy for B2B products. This often engages all decision-makers, helping you understand their needs and objections.
The Letter of Intent is often preceded by other Market Engagement experiments to generate leads.
Tags: B2B
2.11 Provincial
Before launching a new product to the broader market, test it with a small group of users to gauge its viability.
Tips:
This approach is typically applicable to services and physical products. For example, you might open a pop-up shop in a tent before committing to a full network of stores.
For digital products, you'd need to develop the product first. In such cases, consider this approach part of a Go-To-Market (GTM) strategy.
More information: pretotyping techniques.
Tags: Physical products
2.12 One-Night Stand
Offer a basic version of your product for a limited time to gauge interest before fully committing to its development or launch.
Tips:
This approach is typically suitable for services and physical products. For digital products, the product would need to be developed first.
This method is also referred to as a "Pop-Up Store."
More information: pretotyping techniques.
Tags: Physical products
2.13 Infiltrator
Put a sample or even just a box of your product in another store (physical or online) to see if people are interested in buying it.
Tips:
This strategy can be combined with the Fake Door technique, but it leverages explicitly an existing sales channel.
I love the example of the founder of Upwell Labs. Using a shirt from eBay to pose as an IKEA employee, he secretly placed his product pretotypes in an IKEA store. When people bought them, he confirmed the willingness to buy without having his own store.
More information: pretotyping techniques.
2.14 Crowdfunding
Validate a demand for your product by launching it on a crowdfunding platform, such as Kickstarter or Indiegogo. This approach might provide evidence of market demand and help you secure delivery funding.
Tips:
Be careful, as the people investing in your product might not necessarily be your target customers. In such cases, crowdfunding simply becomes a financing mechanism.
3. Minimal Products
The experiments in this group deliver real customer value. They are cheaper to perform than full-feature products but involve higher costs than experiments from the previous category.
Through these, you can evaluate:
Value proposition
Viability (including pricing)
Usability (including messaging; note that the scope of validation using Minimal Products is limited)
All experiments in this group can be considered MVP Prototypes or Pretotypes.
3.1 Piecemeal
Leverage existing tools and platforms to simulate the user experience.
By combining off-the-shelf components and no-code / low-code solutions like Bubble, you can quickly create a functional prototype.
Tips:
Also known as “Impostor.”
Another variant, “Mash-Up,” involves using many existing products to simulate a product experience.
3.2 Concierge
Create a high-touch, personalized service to your customers, simulating the experience of a fully automated product.
Let’s say you're building a personalized learning app. You might start by manually creating learning plans based on a simple form people fill in. This will allow you to validate your idea and understand your customers better before investing in an automated solution.
Tips:
Unlike in the Wizard of Oz, the user is fully aware of your manual work.
3.3 Wizard of Oz
Create an illusion of an automated product.
This will help you validate demand and gain insights into user behavior without investing in developing a full-scale product.
For example, suppose you're developing a chatbot. In the initial stage, you could use human operators to respond to inquiries, emulating an AI-powered chatbot's user interactions.
Tips:
Unlike in Concierge, users are unaware of what happens behind the scenes.
3.4 Single Feature Product
Create a limited version of a product that satisfies only one customer need.
Tips:
In most cases, this is more of a GTM strategy. You don't want to build a product to validate your idea.
4. Feasibility Prototypes
Experiments in this category emphasize feasibility, verifying what's possible with current technology.
Keep reading with a 7-day free trial
Subscribe to The Product Compass to keep reading this post and get 7 days of free access to the full post archives.