10 Tips for Establishing a Successful Test & Learn Environment in the Retail Industry

Decisions about discount rates, product bundles, offers, changes in the store design, process changes and various other investments are part of daily challenges retailers face with every day. For most companies, unfortunately, the “gut feel” is the main driver for many of these. While the value of experience and expert opinion is unquestionable, sentences starting with the phrase “I think…” increase the chance of failure and leave vast opportunities untapped.

You can download PDF version of this whitepaper here.

Conducting business experiments are fairly easy in most industries today. Marketing departments have already adapted to experimenting leveraging their below the line capabilities, developing and testing numerous campaigns concurrently in many industries.

In retail, the use of business experiments is almost inevitable, with factors like the impact of the store experience in sales, number and variety of products sold and short product life-cycles hence the need for continuous change. Considering the number of alternative initiatives a retail company can undertake, the only way out is to continuously test, assess and learn.

By introducing the “laboratory stores” concept, some retailers around the world are now testing new offers, products, even change in their HR policies. Leading retailers with different offerings such as Sears, Subways, and Famous Footwear are among those currently using business experiments to test the effect of different business variables on their bottom line figures.

The chairman of the Sear Holding Edward Lampert describes the urge to experiment in stores quite clearly:

“One of the great advantages of having approximately 2300 large –format store at Sear Holding is that we can test concepts in a few stores before undertaking the risk and capital associated with rolling out the concept to a large number of stores or to the entire chain”

In order to develop test and learn capabilities via business experiments, retailers should:

  1. Get top and mid-level managers on board, create awareness and demonstrate the value add by business experiments
  2. Organize a group of experts who can evaluate and translate statistics into business decisions
  3. Establish a platform to track the performance of test and control stores without any bias and to keep repository of the key learnings for the future
  4. Standardize the business experiment processes and integrate them into core business processes
  5. Share the results, show value add and promote test & learn culture in the company

Here are 10 guidelines for retailers willing to establish a test & learn culture:

Distinguish between what can be tested and what can not

Usually doing business experiments will provide more complete and accurate insights in more tactical decisions rather than high level investment decisions. A well designed test will support a decision on discount rate of an offer but could say relatively limited about a company acquisition, for instance.

Do not try to test hypothesis that are interdependent at the same time

Although there might be quite a number of tests conducted at the same time, interdependencies should be taken into account to really understand the root causes of the business impact. A hypothesis about the shelf location of a product and a hypothesis about the packaging of the same product shouldn’t be tested at the same time since it will not be possible to understand what really caused the final impact; the location of the product or the new package.

Choose test and control stores carefully

Usually companies have tendency to choose the closest or most convenient stores as their laboratory stores. This choice leads to biased results since they usually don’t represent an average store. Decisions should be made by carefully looking at various dimensions like sales figures, customer profile of the store, product range in the store, competitor proximity, employee capabilities and manageability. Try to find representative stores for testing. Once you have identified the test stores, use the most similar ones as control stores.

Effectively track business experiments’ history

Business experiments are meant to learning from. A test result –whether a success or a failure- brings lots of insights for the future decisions. Categorization of those results and storing them in the memory of the company -not the employees’- is crucial for incremental impact of test & learn culture over the years. Make a platform to manage your newly gained knowledge, enabling searching for the definitions, results and key learnings of the previous tests.

Don’t test for the sake of numbers

Although statistical tests and business realities do not necessarily require different designs of experiments, in some cases they do, which requires a fine-balance to get the most out of testing. Testing a new offer on a couple of thousand customers across ten different segments might provide in-depth insights and statistically significant results, but the cost and complexity of such a test may not be justified business wise. On the contrary a handful of observations will not be statistically significant although it might be adequate to make an accurate business decision in some cases.

Standardize the test process and embed it into the business as usual

Create standardized processes regarding the business experiments like selecting the test sample, deciding on measures, and analysis of the results and embed them into the ongoing business processes. Processes like campaign launch, offer development or commission structure changes should refer to test & learn processes at some points. Managers should demand these analyses during their decision making and approvals should be linked to the test results.

Automate for the test & learn culture

Usually one of the greatest challenges while establishing the test & learn culture is the load of manual operations that is required to get a clear test case. For the companies which are commitment to this culture, the ideal solution is the automation of measurement and analysis as part of their reporting environments. This will not only make it sustainable but will also prevent rework and possible data quality issues. Integration of the POS systems with test and learn platform is a good start.

Make sure that the test & learn process is user friendly

Although test and learn process includes statistical or analytical elements, it’s the business departments who really need those tests and will use the results for the companies’ benefit. Translating the language of statistics into business words will remove the barriers and will get more people on board for the business experiments.

Don’t suffer from paralysis by analysis

Speed is one of the must-haves in the retail industry. Companies should make sure that they are not loosing their competitiveness in speed of decisions, while doing lots of tests and analysis. Getting paralyzed with more than enough tests is as bad as following only the guts feel.

Train your employees on basics

Like many things in retail environment, test & learn culture is very dependent on the people. At the very basic level, all relevant parties should be speaking the same language when it comes to business experiments. Even an employee who has no responsibilities regarding business experiments should be aware of terms like hypothesis, control store, sample size or lift of the test. What people don’t feel comfortable with, they become either afraid or ignorant of.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s