The web developer's guide to A/B testing

Default avatar.
September 21, 2012

ThumbnailRunning a web development business isn't an easy endeavor at all. You have to manage multiple projects, tough clients, and a constant stream of relentless deadlines.

One key methodology which enables a web designer to generate more return on investment for his clients’ websites, web ads, email newsletters, offers and promotions is A/B testing.

Don't worry if the term sounds alien to you. It's actually a very simple process; segmenting your users into two different groups and showing each group a different version of your page. To put it in a simpler way, A/B testing — or split testing as it's sometimes known — is showing different page elements to different people and understanding which one gives better sign-ups or sales.

Why is A/B testing important for small businesses?

For one primary reason: to make more money.

As a web designer working with a small business owner, you know that investment by your client has a clear return on investment (ROI) expectation attached to it and successful conversion rate optimization provides one of the best returns on that investment.

To see how it works, let’s take a simple example. Suppose the website has 5,000 monthly visitors with a conversion rate of 4%, meaning 200 people end up buying. If you wanted to increase sales by 50%, you’d have to increase the number of visitors by 50%. Getting the additional 2500 visitors would mean expenditure on online/offline marketing activities that might or might not attract visitors who want to buy.

The second option is to optimize the conversion rate.

You think hard about what your client’s customers want, create a different headline that you feel better communicates the value provided by his business and split visitors 50/50 between the original (control) and the variation (also called “challenger” by some).

After a week when your test has had about 1000 visitors, the software says it’s confident that the variation headline is better and converts 10% more than the original headline. That’s 20 more sales at the cost of 30 minutes to sign up for an A/B testing tool and setting up a test. Not to forget the large smile you’ll be greeted with when your client sees you.

Choices

Image via Shutterstock

This is why the eCommerce world is raving about Conversion Rate Optimization (CRO). The return-on-investment for CRO, through A/B or any other form of testing can be huge. It has been in wide use by large enterprises like Amazon and Microsoft for years but has only recently become easy and affordable enough for web designers and small business owners to undertake on their own.

Choosing a tool

A variety of A/B split testing tools are available and for your initial test, it’s best to go with a free trial version. You can see a good comparison at WhichMVT.

What elements to test

The great part about A/B testing is that you can test almost anything. To get started, you should try some of the following:

  • The Call-To-Action’s wording, placement, size and color
  • Headline or product/service description
  • Pricing and promotional offer display
  • Images on landing pages
  • Copy on product pages
  • Insertion or removal of videos
  • Amount of text (short versus long)
  • Form length and types of fields
  • Link overdose (too many and you run the risk of distracting visitors)
  • Inclusion of Live Chat

This is obviously not a comprehensive list, but is very good place to start if you’re just beginning with A/B testing and want to concentrate on those elements that, in my experience, have shown to give the best results.

Setting up a robust test

It’s extremely important that you run the A/B test in a correct manner to prevent any error creep-in.

First, decide what it is you want to test and on what page of the website. For example, if the client reports that a lot of his customers ask for a certain feature which he does have, you could try mentioning it in the landing page headline. If a lot of visitors come to the signup page but then go to other parts of the website, you could try reducing link and text clutter and keeping the form as the focus of the signup page.

Test the control and variation on the same kind of visitors at the same time. For example, the visitor segment coming in from Facebook, in the week starting 1st July 2012 will be 50/50 split between a landing page with a control and variation headline. Avoid the common error of showing the control for a week and then the variation for the next. The comparison between their two conversion rates is erroneous.

Experiment

Image via Shutterstock

Let the test run its course. Often, you’ll see a positive increase within a day or two and feel great about it. But hold your horses. There’s something called statistical confidence that tells you whether the results you see can be trusted or not. Wait for your chosen A/B testing software to report statistical confidence.

Don’t let the test run for too long. This may result in lost sales as you may be showing some of the visitors a low performing variation. Use an online calculator to decide how long to run a test, most A/B testing software comes with this kind of tool.

If you have changed something that is shown on many pages, ensure that visitors see the change across all pages. For example, if you have changed the color of the price image on one product, a particular visitor should see that same color across your product catalogue. A mismatch could mean inaccurate results.

Understanding the results

A CRO test can have three kinds of outcome; a positive outcome, a negative outcome or a “no result”, i.e. there is no difference between the control and variation. The first time you test, it’s likely that you won’t get a positive result. That’s OK. Testing to optimize conversion rates is not a one hit game. You must do it over and over again, tweaking every page on your client’s funnel to maximize the overall conversion rate.

Decisions

Image via Shutterstock

Many times, it will happen that the results surprise you. What you might consider to be a low performing headline or website design might turn out to be the better performing one. In such cases, it is best to let go of your intuition.

A/B testing isn’t about following your gut; it’s about letting the actions of visitors tell you what’s best for your client’s business.

A few tips

When making changes, make it large. Don’t hesitate to make bigger, bolder changes. This ensures that you get varying results and better insight into what works and what doesn’t.

Headlines are super important. When creating them, work with your client to focus on the benefit that his customers derive from his products/services and not on what he wants to sell. Think from their perspective. Convince your client to test lots of different offers such as a free one month trial, three for the price of two, etc.

Put in a mechanism to collect customer and media testimonials and then add them in the website. These go a long way to providing social proof.

Keep the focus on benefits and not features. The client will probably love his product because of the awesome features it provides, but customers love products for the benefits they provide.

Test, test, and then test some more. After that, test again. Obviously, once your client starts to see a few positive results, you won’t need to do any convincing.

Conclusion

In today's competitive business world, small businesses are using the internet to maximize value and quite often, they do this using the help of a freelancer or an agency. A/B testing is one way in which you can optimize your client’s marketing presence and generate more sales. For maximum effectiveness, this should be done in a continuous manner, ensuring that your clients see you as someone who helps their business’ bottom-line.

Have you introduced A/B testing to your clients? How did it affect your conversion rates over the long term? Let us know in the comments.

Siddharth Deswal

Siddharth Deswal works at Visual Website Optimizer, the world’s easiest A/B testing software. He’s been involved with web development for about 8 years and actively looks to help online businesses discover the value of Conversion Rate Optimization. He tweets about A/B testing, landing pages and effective marketing tips on @wingify

Read Next

20 Best New Websites, April 2024

Welcome to our sites of the month for April. With some websites, the details make all the difference, while in others,…

Exciting New Tools for Designers, April 2024

Welcome to our April tools collection. There are no practical jokes here, just practical gadgets, services, and apps to…

14 Top UX Tools for Designers in 2024

User Experience (UX) is one of the most important fields of design, so it should come as no surprise that there are a…

What Negative Effects Does a Bad Website Design Have On My Business?

Consumer expectations for a responsive, immersive, and visually appealing website experience have never been higher. In…

10+ Best Resources & Tools for Web Designers (2024 update)

Is searching for the best web design tools to suit your needs akin to having a recurring bad dream? Does each…

3 Essential Design Trends, April 2024

Ready to jump into some amazing new design ideas for Spring? Our roundup has everything from UX to color trends…

How to Plan Your First Successful Website

Planning a new website can be exciting and — if you’re anything like me — a little daunting. Whether you’re an…

15 Best New Fonts, March 2024

Welcome to March’s edition of our roundup of the best new fonts for designers. This month’s compilation includes…

LimeWire Developer APIs Herald a New Era of AI Integration

Generative AI is a fascinating technology. Far from the design killer some people feared, it is an empowering and…

20 Best New Websites, March 2024

Welcome to our pick of sites for March. This month’s collection tends towards the simple and clean, which goes to show…

Exciting New Tools for Designers, March 2024

The fast-paced world of design never stops turning, and staying ahead of the curve is essential for creatives. As…

Web Tech Trends to Watch in 2024 and Beyond

It hardly seems possible given the radical transformations we’ve seen over the last few decades, but the web design…