ArtsAutosBooksBusinessEducationEntertainmentFamilyFashionFoodGamesGenderHealthHolidaysHomeHubPagesPersonal FinancePetsPoliticsReligionSportsTechnologyTravel

A/B Testing is essential! adWords, Yahoo, ad text, sales copy, and so on ...

Updated on June 16, 2011

A/b Tests for Ecommerce? YES! For Hubs? Maybe not so much ...

Before we dive into the philosophy, and a few techniques, we should probably cover exactly what an A/B Test is. Once you get the basic concept, the value will be obvious, and you will probably think of many ways it can be applied.

an A/B Test is exactly like what it sounds like, and is named specifically. You are testing A versus B and vice versa. This method is a scientific approach, which is why they use the word 'test'. So, in a basic A/B test, you have:

  1. A desired goal (sale, subscription, subsequent pageview, click, higher CTR, Higher CONV RATE, etc ...)
  2. A portal to that goal (generally a web page that leads to, or facilitates the desired goal).

    • portal / webpage can be a product page, an article, a landing page, and so on.
  3. Two different ideas, or approaches for that portal (which from here on out we will call a Landing Page, for simplicity).

    • The first idea you want to try will be called: A
    • The second will be called: B
  4. A method to determine significant success (measured in 'difference').
  5. A method to keep track of the data, measure it, and produce actionable information.
  6. A method to implement the change(s) inspired by the results of the test.

So, in simpler terms: An A/B Test is a procedure that measures the efficacy of two different approaches to the same goal, with a focus on determining which is better and why.

Just to be clear ... Can I get an example of an A/B Test?

I think that's a good idea. Let's show you a quick example of an A/B Test, so you can put the theory into motion in your mind.

With the primary picture (graphic [top]) for this article in mind, let's say you have a store that sells Mountain Bikes. You're using a variety of methods to drive traffic to your site that sells mountain bikes. So, for this particular test, you want to increase the number of sales for a specific type of bike: The Panther Mountain Bike

The way your store is built, someone must first look at the product page before they can make a purchase (which makes it a required step in our test). Once on the product page they can either: A.) leave your store, B.) visit another page on your site, or C.) purchase the bike. Our goal for this test is to increase the number of people that purchase a bike directly after viewing this page.

The A/B Test would go like this

  1. You would create two product pages for the same exact product.
  2. Each page would be different in some way.
  3. Let's say in this example you change the headline, the product picture, and you move the "buy now" button to a different place on the page. But, you keep the sales copy exactly the same.
  4. The key performance indicator (KPI) in this example is: number of panther bikes purchased
  5. Have a way to measure and record the activity of the test.
    • This is critically important, and we'll discuss this in more detail below.
  6. Launch the test and collect the data.
  7. Once the data is collected, and a statistically significant threshold has been achieved, apply the winner as a permanent change to your site.

What do you mean, 'statistically significant threshold '?

This concept is of particular importance. You need a certain amount of data before you can deem your results "reliable" and "actionable". Let's look at two extreme examples:

  1. After 10 sales, you decide to end and measure the test results: Page A produced 5.45% higher sales than Page B.
  2. After 1000 sales, Page B produce 3.96% higher sales than Page A.

In our above example, after 10 sales, it looked like Page A was the winner, but because your sample was so low (only ten sales) you were not looking at statistically reliable data. If you were to implement the results after only ten sales, in this case, you would have picked the worse of the two iterations. After 1000 sales, even though the increase is lower (only 3.96%) it is much more statistically reliable, and should be implemented on your site as a permanent change (or until another test performs better!).

Why should I A/B Test - What's the point?

The goal of an A/B Test is to identify, scientifically, a more successful approach to your Ecommerce goal. Our web pages, and stores, especially the smaller ones, are put together by web designers, graphic designers, copy writers, and sometimes just regular business persons with absolutely no training in any of the above disciplines.

So, while the one working on the site might think the navigation architecture is brilliant, the product image is stunningly artistic, or that the sales copy was written with such grace that it could make a hardened construction worker question his place in reality, none of that matters if the site doesn't convert visitors into sales. And I'll tell you this: Often times what you think will work doesn't, and what you wouldn't expect to work, does.

That is the primary reason we test. It takes the opinion out of the equation and measures raw performance. So if you're serious about your Ecommerce store, and you're not testing it CONSTANTLY, then you are missing out, big time.

Well, I understand what an A/B Test is, and the theory, but how the heck do you actually perform one? Is there software to do this?

Excellent question. Yes, there are many ways to do this.

One of the most popular, and powerful, ways to do this is to advertise with Google's program called adWords, which if you're reading this article, you should at least be vaguely familiar with if you're not already sleeping together.

There are other software programs out there designed to manage A/B Testing on your site, and they are usually part of an analytics suite. But, for the sake of simplicity, and freeness, if you're aready using adWords to drive traffic to your site, use their A/B Testing.

A/B Testing is just one of several tests you can perform

We're talking about A/B Testing in this article, because it is the simplest and quickest way to get actionable results; Which is better? A or B? Simple. But there are other types, types that get incredible complicated and require throngs of traffic.

If you research this topic further you will inevitably run into the Multivariate Test. The Multivariate Test is also very powerful, maybe in some ways more, but it is not always appropriate. As its name suggests, it is a test that deals with multiple variables. So, lets say you want to  test three different headlines, three product pictures, two 'buy now' buttons, and two paragraphs of sales copy. Each variation is is a variable in the Multivariate test. The test will create every possible combination of the above variables and deliver it to the customers at random. After a significant amount of data is collected and analyzed, the best performing combination can be identified. This method is very intense, and should only be conducted by professionals, on a site with lots of traffic.

So how does understanding A/B Testing help the Hubber?

In many ways, it probably doesn't. But in some ways, it surely does.

Not meaning to be offensive, at all, but many Hubbers are neophytes in the realm of Internet technologies, analytic methods, and optimization techniques. It is not a bad thing, or demeaning, it is just the truth. They bring raw talent to the table, and Hubpages gives them an easy to use venue to publish said content. But I guarantee the Hubpages team tests the sh#$ out of this site. Daily. Especially when they're bringing in about $11,000 per day in ad revenue. Most likely more.

But, what I've found, is that most Hubbers are eager to learn more about the industry and what goes on behind the scenes. Well, here is some of it. If you're a Hubber who also has their own store somewhere, you better be testing it! Or, you can conduct a similar type of A/B Test with your hubs, you'll just have to be a little creative with how you manage your data, and how you present the test.

Instead of wondering why your site or store or hubs are performing the way they are - you can know! You just have to test for it.

As always, just let me know if you have any questions, or would like me to clarify something for you.

Be peaceful on your way,



    0 of 8192 characters used
    Post Comment

    • Alladream74 profile image

      Victor Mavedzenge 

      7 years ago from Oakland, California

      More similar information would be great.

    • rebekahELLE profile image


      8 years ago from Tampa Bay

      TS, definitely, write some more ecommerce hubs. the topic is up for another week with HubMob which is a great way to get read and reach many!

    • Princessa profile image

      Wendy Iturrizaga 

      8 years ago from France

      Thanks for taking part in the HubMob with such valuable information, you really made it sound easy!

    • Time Spiral profile imageAUTHOR

      Time Spiral 

      8 years ago from Florida


      I wasn't sure if this type of information, being fairly technical, had a calling for on HubPages. But I wanted to participate in the HubMob, so here it is.

      Maybe I should put more Ecommerce / SEM related hubs in the pipeline?

    • Green Lotus profile image


      8 years ago from Atlanta, GA

      A big thumbs up TS! I bookmarked this too. You really have a talent for presenting a clear, well-structured Hub. You make A/B testing sound so easy. I'm not ready for Adwords (hope I don't need to go there), but I see how you can skew the same methods here on Hubpages especially within group Hubs. Thanks!

    • janiek13 profile image

      Mary Krenz 

      8 years ago from Florida's Space Coast

      Excellent hub! We are sleeping together, how did you know? lol

    • rebekahELLE profile image


      8 years ago from Tampa Bay

      great info, TS! this is a very well written informative hub for anyone who wants to increase traffic and make sales with their websites, e-commerce stores. thanks for sharing. :]


    This website uses cookies

    As a user in the EEA, your approval is needed on a few things. To provide a better website experience, uses cookies (and other similar technologies) and may collect, process, and share personal data. Please choose which areas of our service you consent to our doing so.

    For more information on managing or withdrawing consents and how we handle data, visit our Privacy Policy at:

    Show Details
    HubPages Device IDThis is used to identify particular browsers or devices when the access the service, and is used for security reasons.
    LoginThis is necessary to sign in to the HubPages Service.
    Google RecaptchaThis is used to prevent bots and spam. (Privacy Policy)
    AkismetThis is used to detect comment spam. (Privacy Policy)
    HubPages Google AnalyticsThis is used to provide data on traffic to our website, all personally identifyable data is anonymized. (Privacy Policy)
    HubPages Traffic PixelThis is used to collect data on traffic to articles and other pages on our site. Unless you are signed in to a HubPages account, all personally identifiable information is anonymized.
    Amazon Web ServicesThis is a cloud services platform that we used to host our service. (Privacy Policy)
    CloudflareThis is a cloud CDN service that we use to efficiently deliver files required for our service to operate such as javascript, cascading style sheets, images, and videos. (Privacy Policy)
    Google Hosted LibrariesJavascript software libraries such as jQuery are loaded at endpoints on the or domains, for performance and efficiency reasons. (Privacy Policy)
    Google Custom SearchThis is feature allows you to search the site. (Privacy Policy)
    Google MapsSome articles have Google Maps embedded in them. (Privacy Policy)
    Google ChartsThis is used to display charts and graphs on articles and the author center. (Privacy Policy)
    Google AdSense Host APIThis service allows you to sign up for or associate a Google AdSense account with HubPages, so that you can earn money from ads on your articles. No data is shared unless you engage with this feature. (Privacy Policy)
    Google YouTubeSome articles have YouTube videos embedded in them. (Privacy Policy)
    VimeoSome articles have Vimeo videos embedded in them. (Privacy Policy)
    PaypalThis is used for a registered author who enrolls in the HubPages Earnings program and requests to be paid via PayPal. No data is shared with Paypal unless you engage with this feature. (Privacy Policy)
    Facebook LoginYou can use this to streamline signing up for, or signing in to your Hubpages account. No data is shared with Facebook unless you engage with this feature. (Privacy Policy)
    MavenThis supports the Maven widget and search functionality. (Privacy Policy)
    Google AdSenseThis is an ad network. (Privacy Policy)
    Google DoubleClickGoogle provides ad serving technology and runs an ad network. (Privacy Policy)
    Index ExchangeThis is an ad network. (Privacy Policy)
    SovrnThis is an ad network. (Privacy Policy)
    Facebook AdsThis is an ad network. (Privacy Policy)
    Amazon Unified Ad MarketplaceThis is an ad network. (Privacy Policy)
    AppNexusThis is an ad network. (Privacy Policy)
    OpenxThis is an ad network. (Privacy Policy)
    Rubicon ProjectThis is an ad network. (Privacy Policy)
    TripleLiftThis is an ad network. (Privacy Policy)
    Say MediaWe partner with Say Media to deliver ad campaigns on our sites. (Privacy Policy)
    Remarketing PixelsWe may use remarketing pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to advertise the HubPages Service to people that have visited our sites.
    Conversion Tracking PixelsWe may use conversion tracking pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to identify when an advertisement has successfully resulted in the desired action, such as signing up for the HubPages Service or publishing an article on the HubPages Service.
    Author Google AnalyticsThis is used to provide traffic data and reports to the authors of articles on the HubPages Service. (Privacy Policy)
    ComscoreComScore is a media measurement and analytics company providing marketing data and analytics to enterprises, media and advertising agencies, and publishers. Non-consent will result in ComScore only processing obfuscated personal data. (Privacy Policy)
    Amazon Tracking PixelSome articles display amazon products as part of the Amazon Affiliate program, this pixel provides traffic statistics for those products (Privacy Policy)