How to Test Something Halfway and Always Fail

As marketers, we must test. Testing is how we tell if something works or not. It’s how we make effective changes to our campaigns, our websites, our pricing, and more.

Testing is also how we find new markets for growth. We test new audiences for existing products, new products for existing audiences, and more.

We test because it is expensive to fail. And when and where there is a way to try something quickly, for less money, we must do it.

HOWEVER….

And this is a big however. Sometimes, the way we test something is wrong. It’s wrong because it doesn’t tell us anything. It tells us the test failed, but we take that to assume that the whole idea was invalid.

To explain what I mean, take a look at this example that I just made up:

Company X has a successful consumer drone business. They make several models and sell them in the US. Looking for growth, Company X believes that they can repeat their success in other countries. But the investment to set up local business units (to “do it right”, their CEO says) is very high. So they decide to “test” a few markets by simply doing the same thing that they’re doing in the US, but expanding the advertising to a few other countries. It’s quick and easy and should indicate whether or not there is demand outside the US.

The problem comes when this fails. Company X now has to ask themselves a tough question. Did it fail because there is no demand outside of the US and we were wrong about our growth prospects? Or did it fail because we didn’t do it right, we didn’t put up the initial investment required to make it work?

Make sense? The point is a simple one, that we have to understand the limits of testing. It is absolutely the right strategy, most of the time. But in order to get the benefits of testing, we must learn to acknowledge its shortfalls lest we fall into the trap of believing false results.