What’s the problem with A/B-testing?

Currently, A/B-testing is very popular, especially among companies working with data bases that have a lot of traffic. This split-run method allows you to easily tweak your products and services.

Still, while A/B-testing is not completely nonsense, there are two major problems that should be considered:

  • Lack of direction for the staff. If you just rely on what works best in practice, your staff will have no criteria to decide whether all their ideas pull in the same direction. You might get a lot of arbitrary suggestions that somehow work out in practice, but overall dilute the image of your product or service.
  • Lack of direction for the user. If your staff has no overall idea, it will be even less likely for the user or customer to understand the big idea behind your brand. As paradoxical as is sounds, a lot of A/B testing might deteriorate the quality of your product.

So what can you do to avoid this? Before starting A/B-testing, you should have a clear vision of what you are heading for with your team. Traditional market research will help you to develop the right overall strategy. Furthermore, you should take a step back from time to time and evaluate the development of your company from the distance. Once again, market research will provide you with the right solutions.

In a nutshell, you should not see A/B-testing as a supplement to market research, but rather as a useful short-term complement for validating the quick implementation of minor tweaks.

Did you like this post? Then your friends might like it as well!

About Florian Tress

Philipp Kemser

Florian is working as a Marketing Manager at Norstat. Feel free to drop him a line at florian.tress@norstatgroup.com.

View all Posts by Florian Tress

RECENT TWEETS