If you’re in the market to put out a new product but don’t exactly know how your customers would react to it, you need to conduct an experiment in a controlled environment. But there’s more to it than throwing a few people into a room with your product and that’s what we’ll discuss today.

Why should experimentations not be overlooked?

A product experimentation subculture exists to be the buffer between development and putting it out on the market. Many organizations always have multiple rounds of experiments and research carried out even before announcing the product to the public.

Experimentation and research should not be seen as a last resort but rather as an opportunity to find defects in the current version of the product, but rather as stepping stones to building that final product that can be as flawless as possible.

When you create a culture of product experimentation, you inspire your product crew to iterate on their thoughts and initiatives. Product experimentation drives product groups to better understand your clients and their desires. When you run common experiments targeted at understanding what your clients will and could now no longer use, you’ll be a few steps ahead in the development stage.

Sounds great, where to begin?

Before you conduct any experiments, there are a few questions you need to ask yourself. 

• What: What is the outcome expected from this experiment? This can also be the hypothesis stage where you note down the desired outcome of the experiment beforehand.
• When: When will the experiment be conducted? It’s a matter of which stage of product development the experiment is to be conducted in.
• How: How is the experiment going to be conducted to get the true results? Remember, the objective of an experiment is not to get the most favorable result, but to improve upon the most unfavorable results.
• Who: Who will be the control group for the experiment? Finding the best control group for your experiment is as essential as any other step as the wrong team can completely throw the experiment off course.
• Why: Why is this experiment being conducted? Could be anything from debugging to obtaining market feedback.
  

How to start the experiment?

1. Keep your clients close 

First and foremost, identify who your existing clients are. Set up a rough outline of their target demographic. This is a double-edged sword as it helps design your product better suited with the customers in mind and 

2. Roll Call!!!

You want to ensure you attain the proper set of clients for whom this experiment is to be conducted. For example, in case you are launching a brand new product in a new geographical area, you would have better chances of obtaining more accurate answers from the people who live/migrated from said area. Picking too small of a sample group 

3. No pride, no prejudice

Hypotheses should be based on psychological principles that are relevant to the situation. Pre-existing biases such as confirmation bias, self-serving bias, cultural and in-group biases to name a few. You must make it clear to the test subjects to keep their biases aside and approach this from a purely factual and functional point of view. 

4. You’re on the clock

You simply cannot continue the experiment for the whole life of a product. You need to decide how long the experiment is to be conducted. Depending on the nature of the experiment and its desired outcome, this duration could range from a few weeks to a few months. Also, consider how the control group’s views and opinions might change over time.

5. Keep your competition closer

Even if it doesn't work for your customer base right away, there is wisdom in what other companies do. Make connections between the trends you observe in your data and how other organizations serving comparable customers are upgrading their products. At best, you'll discover successful ideas that you can expand on, and at worst, you'll discover how your customers are acting differently.

6. Define your yardstick

Now that you’ve identified the competition, let's define the metrics we'll be tracking during the experiment. Metrics are measurable data points that every product manager must keep track of in order to assess the performance of an item or idea.

Defining and tracking the right product metrics allows you to make the best product decisions and take the best actions in the future. So, when you run an experiment, make sure you outline all of the crucial metrics you want to track, as well as when you'll declare your experiment a success.

7. Have professionals on board

Sometimes we might think we’re smart enough until someone smarter comes around. In this case, hiring a professional researcher can truly benefit your experiment in more ways than you can imagine. Not only will you receive a fresher perspective on things but also a professional touch to common data that might be overlooked or even considered by non-professionals.

8. Observe 

When conducting the experiment, maintain regular notes of observations. These could be anything from small comments to larger grievances. The key is to read between the lines. Sub-conscious or first reactions of your sample can give valuable information and insight. This will be taken care of by the professional you hire but it’s always good to have and record your own personal observations.

9. Broaden your horizons

Alternate between high-risk and low-risk trials, large and small in scale, and aimed toward learning and impact, to maximize resourcing, promote learning, and provide a consistent workload. This variety guarantees that your team has a mix of learning opportunities and you get more information and data on different topics.

10. Embrace failure

Accepting failure is an important part of product development: if your team is afraid of failing, they will not undertake new experiments or test out new ideas. To obtain a better understanding of your clients' needs, your team must be willing to take risks that can be recovered. A failure occurs when a change you made has a negative influence on the customer experience, as experiments measure the impact of product modifications on the customer experience.

To wrap it up…

The whole point of your experiment is not to get the right answer. Be sure to stay within the legal limits while conducting any experiments and take your test subjects’ state of mind over time into consideration too. If you’re looking to take the A/B test, here’s an article that discusses the Top 10 A/B Testing Tools for Your Business.