Statistical Significance Calculator

Use this statistical significance calculator to determine which of your ads perform better in your A/B split tests. This was built to be used with conversion data from AdWords ads, but you can obviously use it for any other type of split test you are running.

This calculator will tell you if there is enough data to declare a winner between two ads. In statistics, a 95% level of confidence is considered to be statistically significant. For your purposes, you may be satisfied with a lower level of confidence and be ready to start another split test.

I especially encourage you to start a new split test if your conversion rates are almost identical and the confidence level is very low. You should be trying to write ads that blow your other ads out of the water. If you ads are performing the same, just pick a winner and move on to something new.

    • Number of Clicks
      Number of Conversions
      Conversion rate:
      Ad A
      10%
      Ad B
      12%
    • Your results:

      Ad B converted 20% better than Ad A. You can be 92% confident that Ad B is the better performing ad.

      This is not a statistically significant result (statistical significance is reached with a 95% confidence level).

One response to “Statistical Significance Calculator”

  1. Quora says:

    How do I properly A/B test two AdWords ads, when one is new, and the other is old?

    The proper way to do this is to start the split test from the point in time when the new ad is added. Ignore all previous data from the old ad. When comparing ads that ran at different times, other variables can cause the ads to perform differently (ch…