Why A/B Testing Is Essential To Your Marketing

by Nathan Yerian

Director of Strategy,
Adhere Creative

Marketing is a discipline in which art meets science. Beyond writing great copy, creating strong visuals, and developing a plan for promotion, companies need to commit to best practices that optimize conversions. The average website converts less than one percent of its traffic into tangible leads, so marketers must focus on critical details like button colors, email subjects and landing page visuals to increase conversions. Also known as split testing, A/B testing provides structure for marketers to analyze and maximize a campaign's impact. Following an experimental approach, A/B testing is essential for a business' quantitative marketing goals.

What is A/B Testing?

As the name suggests, A/B testing involves a comparison between two groups, A and B. Groups A and B are identical except for one independent variable, which can be copy, color, link placement, email subject line or something else that the marketer wants to study. Using a controlled experiment, marketers can compare two versions of a variable to determine which performs better in terms of concrete and measurable results like click-through rates, newsletter open rates, survey response rates or conversion rates. Marketers may choose to examine one or several outcome metrics in one study. To be effective, marketers must test groups A and B simultaneously. Otherwise, extraneous variables like seasonality, time of day, day of week, media trends and consumer patterns will be impossible to control and may contaminate results. For an effective experimental design, marketers should plan A/B tests ahead of time and clearly define the independent variable of interest. As part of the A/B test's initial design, marketers should carefully select quantifiable outcome measures for gauging success.

Testable Elements

A/B testing is flexible enough to analyze a variety of marketing elements. The following provides a snapshot of commonly tested options for websites, landing pages and email campaigns:

  • Email subjects
  • Page headlines
  • Advertising copy
  • Call to action buttons
  • Color schemes
  • Link placement locations
  • Site layouts
  • Key graphical elements
  • Descriptions of products or services
  • Checkout features
  • PPC (Pay Per Click) advertisements

To run a straightforward A/B test, marketers should manipulate one variable per experimental design. Otherwise, results may interfere with one another. For instance, imagine an experiment that simultaneously tests advertising copy and color schemes. Group A clearly demonstrates higher conversion rates than group B; however, the results are less than straightforward. What caused the outcome — the colors or advertisement copy? From a quantitative perspective, the answer to that question is inconclusive.

Available Software

Marketers can design A/B tests using a variety of free and paid analytics platforms.

  • As a free option, Google Website Optimizer caters to analysts of varying skill levels. Marketers can choose different combinations of design elements, headlines, images and text to test. By randomly selecting what site visitors see, Google will test different groups to determine which properties yield the highest conversions. This Web-based product comes with a visual reporting interface to accommodate varying math backgrounds.
  • Adobe Test&Target is part of a paid digital marketing suite originally developed by Omniture. This resource helps marketers test a range of campaigns from websites to email newsletters and display advertisements. Robust features like real-time click and revenue reporting are also available to help businesses make quick decisions across campaigns.
  • HubSpot provides a comprehensive, subscription-based marketing platform with tools for blogging, social media, SEO, website management and analytics. Using HubSpot, marketers can implement advanced A/B tests across key marketing elements an analyze results.
  • Optimizely is a subscription-based analytics platform that provides real-time data. The service advertises multi-browser testing and automatic goal monitoring. With both a graphical user interface and code editor, features cater to varying Web developer skill levels.

Best Practices for Experimental Design

In addition to clearly outlining goals, defining elements for testing, and conducting controlled experiments, marketers should follow basic principles of research. First, the visitors that see groups A and B should be randomly selected with no predictable pattern. Second, groups should be large enough to make a statistical claim. Groupings of fewer than 30 people may not produce reliable results. When implementing the test, analysts should determine how they want to split the groups. Typical splits are 50/50, 90/10, and 80/20. If sample sizes are unequal, marketers should compare groups based on proportions rather than count data.

Most Important

While requiring structure, A/B testing allows for creativity, and regardless of whether the analysis comes from the perspective of an artist or experienced statistician, the findings can add value to a company's long-term marketing strategy. Today’s marketing environment requires consistent improvement to stay ahead of the competition. A/B testing gives you the ability to improve your marketing components in a proven and measurable way.


No Comments

No comments yet.


Leave a comment