Our Work
Why Us
Blog
Pricing
Book a demo
Blog

Why Performance Marketers Need to be A/B Testing Designs

Andres Levinton
Andres Levinton
9 min read

Is there a time that you shouldn’t test a marketing campaign’s ad creative? This was a question raised by Amrita Mathur, VP of Marketing at Superside, at one of our Ad Designpalooza sessions. The panelists, Shanee Ben-Zur, CMO at Crunchbase, and Hakim Garuba, Performance Marketing Manager at Square, agreed that while testing is important, there are some cases when it simply doesn’t make sense.

Here are some quick testing rules to follow:

  1. Sample Size Matters: You shouldn’t test the ad creative when the sample size is too small. Small sample sizes don't provide enough information to draw insights from.
  2. No Budget, No Test: If there’s not enough budget to see through your experiment to acquire the right amount of data, then what’s the point?
  3. Stakeholders Must Be Aligned: For creative a b split testing to be successful, stakeholders need to agree on their standards of success and what the main KPIs are.
  4. Good Measurement is Key: You need to be able to accurately monitor and measure your data. Without good (clean) data, you can’t trust the results of your A/B test.

As performance marketers, A/B testing or split testing is not a new concept. It’s a marketing tactic that we often use to enable us to make data-driven decisions, especially in digital advertising. We research, analyze, hypothesize, test, launch and the cycle repeats itself. If executed properly, the results can have a positive impact on our bottom line.

The Ultimate On-Demand Ad Design Event!

Join 5,000+ marketers and creatives who have already watched the recordings! 6+ hours, 11 epic sessions, 15 rockstar speakers.

Watch on-demand

There have probably been many times where you redesign a banner ad, revise a landing page design or make minor tweaks based on trends or personal preferences. The truth is this method doesn’t always work, especially if you’re looking to scale, test, and iterate complex designs. Being a performance marketer often means finding the optimal mix of offer, copy, and design. Therefore, if you're not split testing or landing page a b testing your designs, are you really optimizing for performance?

Why is A/B Testing Important in Design?

A/B testing is an important evaluation tool in the design process. The design process isn’t just an art; it’s also a science that relies on cold, hard data from the research and ideation stages.

Here’s why every performance marketer needs to be testing their designs:

  • Running A/B tests allows performance marketers to take the testing results and share that newfound information with their designers.
  • This user behavior data gets fed back into the design process to hypothesize, test, launch, and iterate, spurring new creative ideas and approaches.
  • And overall, A/B testing is an effective tactic to use especially if you involve your designer throughout the entire process. Better ROI, anyone?

TIP: Your designers shouldn’t be just task takers— the real magic happens when designers are given the chance to be strategic, creative thinkers. But they can’t do this without knowing how their designs are performing, and what the campaign goals are.

The 4 Main Benefits of A/B Testing

1. Improved Content Engagement

A/B testing forces you to evaluate every aspect of your creative. This means that while creating multiple variants, you’re also creating a list of potential improvements that may be statistically significant. Consequently, using an a/b testing tool makes the final version of the page or creative better for your target audience.

2. Increased Conversion Rates

A/B testing may be a longer process, but when executed properly, will help you improve clickthrough rate, convert more leads, and ultimately, increase sales. Better customer engagement means more customer conversions, which leads to increased sales volume for your business.

3. Ease of Analysis

With A/B testing, it’s simple to analyze real, factual results—no more guessing. The data helps you determine a “winner” and a “loser” based on straightforward metrics (i.e. time spent on page, conversions, etc.). There’s no ambiguity or room for swaying your opinion over the data. As you know, the most beautiful designs aren’t always the most effective, so A/B testing lets the data speak for itself!

4. Reduced Risks

A/B testing can help you examine website visitors and audience behavior on your creative before committing to major decisions and significant company budget.

We should also mention that there are a few downsides to over-testing your creative. Although testing designs can often lead to better outcomes in the future, there are times when it doesn’t make sense to test at all. If you’re keen to learn more, we’ve got a whole blog on how to balance your data and creativity! Use the data wisely—the data should work for you, you should not work for data.

How to A/B Test Design at Scale?

A/B testing a handful of variants is manageable, but have you ever tried to do it for dozens, if not hundreds, of variants? Let me put it into perspective for you:

As the Performance Marketing Manager at Superside, I recently launched a Facebook ad campaign to test which ad creative resonates with our target audience. We ended up with 36 design variants and two different text elements with a total of 72 ad variants. Thankfully for us, we have a scalable design team to help us execute this. This may not be the case for you.

Here’s what you can do to A/B test design at scale:

1. Improve Internal Processes

It’s important to involve your design team throughout the design process, which includes reviewing results from your tests. This helps your designers understand what you’re testing and why, so that they can constructively contribute with ideas on what to test next.

Advantages:

  • Commitment: You receive an upfront commitment from your design team to contribute to the test by developing the iterations.
  • Goal alignment: With your design team involved, you can ensure your standards for success for the test are aligned.
  • Understand Return On Ad Spend (ROAS): A/B testing helps to evaluate and efficiently allocate your advertising dollars.

Disadvantages:

  • Resource intensive: You may not have the humanpower or budget to execute a successful A/B test.
  • Prior commitments: A/B testing can pull your design team away from higher priority tasks.

2. Review External Resources

If you don’t have the internal resources to scale design, this is an opportunity to work with agencies, freelancers, or a solution like Superside who can build creative iterations and tests for your campaign.

Advantages:

  • Time to hire: Work with talented, quality designers without the entire hiring process.
  • Faster delivery: For example, when you work with Superside, we offer 12-hour turnaround times.
  • Cost-effective: Cost varies based on who you work with, whether it’s freelancers for one-off projects, or tapping into external design teams like Superside at ⅓ of typical agency costs.
  • Reliability: Work with an agency or freelancer with a good track record of projects similar to yours
  • Easier: The briefing process can be easier. Superside has a 10-minute brief.

Disadvantages:

  • Limited capabilities: Depending on your design needs you may have to work with several parties. This isn’t the case at Superside – we have a full-stacked team with a range of skills sets to execute your design needs.

3. Build a Strong Tech Stack

Review and acquire tools that will make A/B testing design at scale easier for you and your design team.

  • Google Optimize is a free website optimization tool developed by Google that helps marketers and webmasters increase visitor conversion rates and visitor satisfaction by A/B testing website content.
  • Superside offers a design management platform that is free to all subscription customers. It lets you easily collaborate with design previews so you and your stakeholders can share and comment on design files, organize and share all your brand assets, and stay connected with your team on ‘Superchat’ to exchange files, files, and feedback.

The Definitive Guide to Digital Ad Design

Get industry best practices and case studies for advertising on: Facebook, Pinterest, Instagram, YouTube, Amazon, Google Display and some key emerging channels.

Get the guide

Best Practices for A/B Testing Design

Now that we’ve covered most of the basics of A/B testing design, the last question we want to ask is how do you conduct efficient A/B tests for design? We spoke with Superside’s Amrita Mathur, VP of Marketing and Anneke King, Executive Creative Director, about their best practices when planning and carrying out A/B testing, and here’s what they shared with us:

Define What Creative Elements You Want to Test (+Metrics)

To start your A/B testing on the right foot, determine what variable(s) you want to test – copy, graphic, layout, etc. You can test one variable at a time or test more than one variable (known as a multivariate test) depending on the complexity of your A/B testing. It’s also important to have key performance indicators (KPIs) defined and baseline results to compare to (ex. current conversion rate).

If you haven’t already, loop in your designer or design team at this stage. As previously mentioned, Superside recently launched a Facebook ad campaign with 72 ad variants. When we knew we’d be A/B testing, we looped in Piotr Smietra, Marketing Creative Director, to get his input on what variables we should be testing and an idea of what the timeline would be to deliver those 72 ad variants.

Create a Strong Hypothesis

A solid, user-centered hypothesis is a must for your A/B test. A strong hypothesis can keep your evaluation focused on the relevant metrics rather than filling gaps with assumptions. Without a strong hypothesis, you can easily determine if an A/B test is even worthwhile to make and test those variables.

A few months ago, Superside conducted an A/B test on our top of funnel landing pages. This was our hypothesis: “If we change our landing page background to a more lively/colored background, then we’ll see an increase in conversions, because the webpage is more appealing to our audience.” And the results? Our hypothesis was true, so from now on you’ll be seeing more lively landing page backgrounds from us!

Failed Tests are a Good Thing

In an ideal scenario, your A/B test has a “eureka!” moment where there is a clear winner and loser. Sometimes, A/B tests fail to produce significant results. If that happens, run your experiment longer. If your hypothesis is disproven, don’t be discouraged. Even a failed test can share insights and opportunities for your next iteration, only increasing your chances of succeeding.

Create a Balance Between Design and UX

When you focus too much on “data-driven” design, user experience can be negatively impacted. A/B testing tends to focus on the quantitative data, but it’s also important to set qualitative KPIs (ex. registration, form submissions, etc.) when running an A/B test. A high-traffic landing page may sound great, but if the page isn’t receiving form submissions, then something needs to be changed to positively influence the user.

Give Experiments Sufficient Time

Conducting an A/B test in a shorter amount of time won’t get you your results any faster. That’s why it’s important to give sufficient time to conduct a thorough A/B test with enough participants or iterations to achieve strong, concrete results. As the saying goes, “slow and steady wins the race!”

Conduct A/B Testing Regularly

Things change over time and that applies to A/B testing. Make A/B testing a regular part of your process to ensure that you have the most up-to-date data on your design variables. A high-performing CTA button won’t always be “high-performing” and that’s when you’ll need to A/B test to see if there’s a better way to do it.

Every marketing team has a method to A/B testing. At Superside, we believe in the process of iterating and optimizing to understand our customer base and market at large. From ABM to content, our marketing team A/B tests design regularly. It also helps that we have a design team and creative support to help us execute (yes, we use our own solution as well!).

Get Frictionless, Delightful Design

From ad creative, to email and landing page design, Superside works with brands like Facebook, Amazon and GroupM to help them scale design better, faster and cheaper.

Get started

Start A/B Testing Design Today

Take this as your sign to start A/B testing design, whether that’s landing page designs or digital ad designs. The results of your creative tests provide valuable data that gets fed back into the design process to iterate over and over again until you have the winning variant.

For performance marketers who want to A/B test design at scale, these are the three main points to remember:

  1. Involve your designers throughout the creative process.
  2. Work with agencies, freelancers, or an advertising design service like Superside to help with the design workload.
  3. Invest in the right tools to make A/B testing easier and more efficient for your team.

If you have started A/B and multivariate testing design, keep going because you’re on the right track!

Published: Aug 22, 2021
Andres Levinton
Written by
Andres Levinton
Andres Levinton is a Performance Marketer at Superside. Besides advocating for quality design at scale, he spends his time playing bossa-nova, searching for waves and cooking Argentinean asados.

Join our community of 5,000 strong and receive the best design and marketing content, biweekly

No charge. Unsubscribe anytime

Continue reading

iOS 13 Update Advertising on Facebook

How Apple’s iOS 14 Update Changes Advertising on Facebook

Hint: great creative is will be the name of the game. The Apple iOS 14 update doesn't need to be all doom and gloom for advertisers—we've got tips for success!

Branding package outline and tips

What’s Included in a Branding Package?

Whether you're refreshing an existing brand or starting from scratch, you need a branding package. We cover what a brand package includes, plus premium add-ons.

How to write a design brief

How to Write a Design Brief (+ Free Template)

Learn about what to include in a design brief, along with some tips on how to properly write one. Or, just access our free design brief template!

Hassle-free design starts here

Superside is an always-on design company that makes design frictionless and hassle-free for marketing, sales, creative and product design teams. This means top quality designs at lightning fast speeds, improved velocity and go-to-market and completely secure and confidential file sharing and collaboration.

In this one-on-one live demo, you’ll see:
- How Superside works
- A first look at the Superside platform
- The most suitable subscription plan for you

Get ready to join 140+ scale-ups and enterprise teams doing good design at scale with Superside’s dedicated team model.
Want to start a design project instead? No problem, get started here

Book a call with us

First name
Last name
Email
Company name
Company size
Phone number
We need your phone number for the demo. We'll never use it for any other purposes.
Book a demo

Company

Learn

Book a demo
Superside is a revolutionary way for businesses to get good design done at scale.
Trusted by scale-ups and enterprise teams, Superside makes design hassle-free for Marketing, Sales, Creative and Product Design teams by combining top designers and project managers from all over the world, with purpose-built technology and the rigour of designops. Since inception, Superside has been a fully remote and distributed company based in Palo Alto, California, with 180+ employees working across 50+ countries and 19 timezones.
Privacy Policy