January 5, 2023
Messaging

What is copy testing: Evaluate & optimize messaging effectiveness

Copy testing uncovers how well your messages resonate with your target customers. Learn how to do it with our 4-step framework.

Competing on features alone is no longer a winning strategy. To stand out from the competition, you need to differentiate yourself in your messaging. Your messages must fit the market, aligning with your target audience’s goals and addressing their pain points.

Copy is a critical element–even twice as important as your design. After testing 36,928 page variants between March 2019 and March 2020, Unbounce found that, across all industries, copy has twice the influence to convert over design.

The only way to know if your messages land successfully is to test them against your audience.

In this article, we’ll review what copy testing is and how automated copy testing elevates your messaging, provide Wynter’s framework for copy testing, and highlight examples of copy testing with Wynter.

What is copy testing?

Copy testing puts your marketing messaging in front of your ICP and collects feedback on how it resonates with them. It provides fast and affordable qualitative insights to refine your value proposition, offers, and landing pages.

Done right, copy testing helps you identify which messages lead your website visitors to take desired actions using both qualitative and quantitative feedback. 

Back in the day, “pre-testing” measured an advertisement’s effectiveness using consumer feedback. Marketers would gather consumer juries, show them ad variations, and measure elements like likeability, persuasion, and day-after recall using both quantitative and qualitative research methods. 

Modern copy testing follows the same principles. You gather a panel of 15-30 people in your ICP, expose them to your website copy, and ask them open-ended questions about your copy’s effectiveness. 

Qualitative vs quantitative feedback

Quantitative data uses numerical data to measure audience response using collection methods like a Likert scale or closed-ended questions.

Qualitative data is non-numerical, describing the attitudes, feelings, or experiences of your ICP using collection methods like open-ended questions and focus groups. 

Combining qualitative and quantitative research improves your data’s integrity; the strengths of one data type balances out the limitations of the other.  

For example, Zingtree blended qualitative and quantitative data when testing their homepage copy with Wynter. They asked their target customer base, Customer Support Directors, to rate the overall clarity of the page. 

Screenshot of zingtree homepage tested by Wynter using qualitative and quantitative methods
Wynter tested Zingtree’s copy using qualitative and quantitative methods 

Zingtree scored 3.9/5 for overall clarity, but without qualitative insights, there’s no “reason why” behind this.

To do this, they asked the question: “after reading everything, what’s still unclear?” 

Zingtree’s audience found the message was clear, but only after doing some digging. The product’s use case was buried so far down the page it wasn’t obviously apparent to visitors. 

Feedback from their ICP included:

  • “The first page was a little vague and could use more description of what is being sold.”
  • “Is this a stand-alone platform? Is it integrated into other current working stations or CRMs?”
  • “After reading the page since I was like 80% understanding, but I have to read the whole page to get to it!”

Mixed-method feedback enriches the data, giving a full picture view and allows marketers to uncover exactly what must change and why. 

How automated copy testing with Wynter elevates your messaging

Automated copy testing is a new copy testing model that tests variations of your company’s messaging with a panel of B2B professionals in your target audience.

Traditional copy testing can take weeks or months. You gather your panel, ask them questions about your messages, analyze the data, tweak your website copy, and test again. By this point you might have lost customers along the way who may not resonate with your new messaging. 

Automated copy testing gives you a head start, putting your message in front of your target customers, with results in under 48 hours so you can apply the feedback on your website as soon as possible. 

Wynter helps you:

  • Collect data on how your audience perceives your messaging. Is anything confusing or unclear? Is there something that doesn’t make sense or misses the mark? 
  • Define the benefits that resonate the most with your audience. What benefits are they most interested in? What benefits are they least interested in? 
  • Validate your audience’s pain points. Do they feel your company understands their needs and solves their problems? 

Wynter uses a 5-point scoring system to test for:

  1. Overall clarity: Is the message you convey clear for your audience?
  2. Interest in the demo: Does your audience feel compelled to sign up for a demo? 

For example, we recently tested OpsLevel’s homepage. 

OpsLevel, a SaaS startup, helps development teams organize and track their microservices in a centralized portal, but does their copy convey this? Our targeted panel of 15 back-end, full-stack, and software developers found that OpsLevel’s homepage lacked clarity on what services they provide.

Questions arose like “how long will it take me to go from zero to a dashboard of useful monitoring information?” or “what is the exact functionality of the service?” 

Other doubts appeared like how OpsLevel differs from the competition. While one panelist found the app’s design and features compelling, they questioned the distinct advantage of using the product. 

Screenshot of OpsLevel homepage tested by Wynter 
Wynter’s copy test for OpsLevel revealed a need for more clarity

Now OpsLevel has centered feedback about how to tweak their messaging to better appeal to their target audience. 

Outsourcing your copy testing cuts down on time and cost, which is valuable for all businesses but especially smaller B2B companies lacking the scale and budget to complete their own testing.

Other tests like A/B testing won’t cut it, especially because B2B companies lack the necessary traffic to conduct a successful A/B test (± 500 conversions/month). The only way to test is through your messages. 

Copy testing vs A/B testing

Marketers frequently blur the lines between copy testing and A/B testing. While A/B testing informs which version of an element (e.g. landing page variation, headline, etc.) worked best, it doesn’t tell you why it worked. 

A/B testing is also limited, with only 1 in every 7 A/B tests resulting in a statistically significant winner. While A/B testing can help maximize ROI, you’re still paying for the losing variation.

Budget constraints aside, you still face questions like: 

  • How many A/B tests won’t reach statistical significance?
  • Do we have more than 500 conversions a month to get an accurate depiction?
  • How many false positives cost money once implemented?
  • Most importantly, why did the winning version win? 

Implementing new copy also risks inconsistent brand messaging. 

Let’s say you’re testing a new features section on your homepage. Through A/B testing, you find that variant B (the treatment) wins.

But what happens if the copy on your product page still quotes the original features of variant A (the control)? Or your social media bio still reflects unrelated features or benefits? A/B testing can confuse audiences with inconsistent messaging.

Copy testing provides what A/B testing lacks: insight from the people you want to click the buy button. 

A reliable framework for copy testing

At time of writing, 53% of users wait no more than three seconds before leaving your site (according to Google Consumer Insights).

As Peep Laja says, in the B2B SaaS world, people want you to get to the point. 

Copy testing helps you refine your message, increasing the likelihood that your ICP will stick around and continue their journey with your brand.

Step 1: Set your research goals

During the initial stages of copy testing, you must decide key details like:

  • What pages or sections need testing?
  • What are you hoping to achieve? Do you want to attract more free trials or leads?
  • How much can you afford to pay panelists for participating in the study?

For example, heatmaps may have historically revealed that a disproportionate number of customers engage with product descriptions but rarely book a demo. If so, your copy may need re-aligning closer to your ICPs needs.

Write open-ended (qualitative) and closed-ended (quantitative) questions. Here, we ask our panelists to rate the overall clarity of a message and their interest in following through on a demo. 

Screenshot of Close Ended Questions
Closed-ended questions provide great quantitative insights

We also ask questions like:

  • What’s your biggest doubt about the offer?
  • After reading everything, what’s still unclear?
  • Why choose this company over the competition? 
Screenshot of Peep Laja tweet about open-ended questions
Open-ended questions allow for qualitative research

Asking both question types gives you a concrete number (e.g. 3.9/5) and provides a detailed explanation to back it up.

Write as many questions as you need. There’s no limit to what questions you might ask as long as they suit your research goals.

Step 2: Find your panelists

Your plan should ideally be made up of 5-15 members of your ICP. Although a single individual in your target market can offer accurate insights, you need more to catch the outliers.

As Peep says:

“Studies consistently show that 5 to 15 users will find 90% of your usability problems. 15 will find 97% of your usability problems. The same goes for copy.”

One person saying “this is bad, I don’t like this” doesn’t necessarily mean you should throw away your messaging. If you have 5-15 people and the issue recurs, it needs fixing. You need enough people to validate the results in your study.

Find people in your target audience who aren’t yet customers so they can offer unbiased feedback.

For B2B companies, your target audience might hang out on LinkedIn or Twitter. Search your product’s use-case to find leads already talking about your offering. 

For example, if you offer a social media management tool, search “social media scheduler” or “social media scheduling tool” and you’ll find leads talking about your offering.

Screenshot of Kenneth Lang LinkedIn Post 
Use social media to find qualified panelists for your copy testing

Alternatively, we’ll help you get your messaging in front of the right audience for you.

Step 3: Conduct research sessions

Now it’s time to put your copy to the test. 

Run sessions in various ways:

  • a one-on-one video conference session with screen share;
  • a phone call;
  • an email.

Although a focus group saves time because you can interview all panelists at once, individual conversations allow your participants to give unbiased answers, uninfluenced by their peers. 

In the research session, either read aloud the questions you wrote and ask them to respond or send them an email with an attachment of the copy you want them to review along with the questions you want answered. 

Step 4: Analyze the data and adjust your copy

Compile your data in one place so you can search for themes within the feedback. We recommend using a simple spreadsheet like Google Sheets or Microsoft Excel. 

Screenshot of Wynter’s message testing data compiled in a CSV
An example of Wynter’s message testing data compiled in a CSV.

If your panelists reviewed multiple pages, give each page its own sheet within the spreadsheet so you won’t mix up feedback. 

Then, use these insights to adjust your copy to match your target audience’s feedback. If you’re wondering how, here’s how Wynter would do it using our message testing method. 

How to transform audience feedback into compelling copy

Each copy test with Wynter goes like this: 

  1. B2B professionals choose their audience based on job title and three different seniority levels
  2. They specify audience details like their preferred employee count and the industry
  3. They configure the test, choosing between a 15 or 30 person audience and decide if they want a CRO professional to compile the message testing results
  4. They set up the test creative by entering in the URL of the page or by uploading an image
  5. They customize the test questions by choosing our pre-made questions or writing their own

After 12-48 hours of private deliberation among our B2B panel, we send the test results. But once you receive the results what do you do with them? 

Here’s an example of how Wynter message tested Qualified’s homepage.

Qualified

Qualified’s product helps marketers convert website visitors into sales pipeline. However, their original copy lacked clarity and muddled their messaging.

Their homepage header introduces visitors with “Welcome to the Pipeline Cloud.” It leaves us wondering, “what is a pipeline cloud?”

Screenshot of Qualified Old Homepage Header
Qualified’s homepage header may confuse their audience

Peep Laja searched all over LinkedIn and Twitter for “pipeline cloud,” but only Qualified’s employees were talking about it. 

We put this copy in front of a panel of marketing professionals using two quantitative questions, how compelling was the pitch (3.7/5)? And what was the overall clarity (3.9/5)?

Validating these quantitative findings, we asked panelists to go into detail and provide reasoning behind their responses.

They shared comments such as: 

  • “After reading the paragraph I am interested to learn how this is different from a tool like 6sense or any other ABM platform. The paragraph was a little vague and if someone doesn’t understand what ABM is it would not make sense to them.”
  • “No idea what pipeline cloud meant originally, but after reading the description below it made much more sense.”
  • “I'm interested, but ‘Pipeline Cloud’ doesn't mean anything to me at this point, so I would consider another enticing headline.”

Qualified’s headline, while clever and a trademark of their own making, lacks clarity. If customers must read beyond the header and subheader to find out what your company does, you’ve lost them. 

The subheading also has so many words that even the Hemingway editor finds it hard to read: 

Screenshot of Hemingway Editor

Using these gathered insights, we helped Qualified tweak their homepage copy to clearly communicate the outcome their product offers:

Screenshot of Qualified’s New homepage copy
A more accessible version of Qualified’s homepage copy resonates better

We removed the confusing “Pipeline Cloud” copy and simplified the subheading so people know exactly what Qualified does within a few seconds of reading the copy. 

Mayple

Mayple offers hiring services for on-demand marketing. We tested their copy with ecommerce and SaaS founders.

While Qualified used many words in their subheader to convey a lot, Mayple doesn’t offer enough. It’s too vague. People may read it and wonder if they’re an agency or offer a freelance marketplace.

Mayple didn’t lead with a market category. We’re left assuming what their product is (a cardinal sin of product positioning):

Screenshot of Mayple’s old homepage copy
Mayple’s homepage copy was vague and confusing

We asked two quantitative questions; how compelling is this offer (2.9/5), and please rate the overall clarity (3.5/5). 

So, Mayple scored low in both categories. Backing it up with qualitative insights, we know why. The panelists said:

  • “I'm slightly confused about what the offer/service is. Working with marketers to grow my business is so broad. Then I scroll down and now I'm wondering if it is about hiring employees. Am I hiring a consultant, freelancer, or employee? Not sure.”
  • “Top vetted is highlighted, and it's backed up by the ‘5%’ in the paragraph below. However, it doesn't tell me anything about HOW they're vetted–not even a bullet point.”
  • “What defines the top 5%?  How many people and where does the pool come from?  How is it judged?  How can I learn more without scheduling a demo?”

Their header and subheader lacks specificity–who are they for? How do they do it? 

Since this test, Mayple successfully changed their homepage copy: 

Screenshot of Mayple’s New homepage copy
Mayple rejigged their copy for a more specific approach

They listened to their ICP’s feedback and included a pre-header sentence to call out their market category. Now visitors know they help companies hire marketing talent.

Then, they define who you can hire with their services: freelancers and boutique agencies. Visitors understand that marketing talent is vetted through results and not sales pitches, resulting in a clearer and more compelling offer.

Chili Piper

Chili Piper helps sales teams schedule sales appointments, but their copy leaves readers uninterested in their offering.

We tested their copy with a panel of Demand Generation and Growth Marketing Directors. 

Screenshot of Chili Piper’s Old homepage copy
Chili Piper’s homepage copy was not convincing enough

They scored a 3.5/5 for overall clarity, with some panelists wondering how the product works and what it does (besides converting or qualifying leads into meetings). 

Chili Piper also scored low for interest in demos with a 2.5/5 overall. Panelists didn’t trust the efficacy of the tool; there’s no case study to read or price to review. They said:

  • “Not really a doubt, but there's not a real ‘offer’ here in my mind–as in, there's no white paper to download or case study to read or price to review.”
  • “Not really clear on individual benefits to each of the solutions or if it would be relevant to my specific sector”
  • “There is zero credibility throughout the landing page I viewed. There is also zero context on the landing page. I was left with marketing-like jargon and messaging without actually learning anything special or unique about the product.”
Screenshot of Chili Piper’s New homepage copy
Clearly stating their offering and audience vastly improved Chili Piper’s homepage copy

Based on readers feeling their offer lacked efficacy, we suggested they lead with a benefit statement so it’s crystal clear why users need Chili Piper. 

We also suggested they call out their target audience of B2B revenue teams and follow it up with a clear outcome or job-to-be-done; “to book with prospects as soon as they express interest in a meeting.” 

Their new copy reduces as much purchase friction as possible. Leads understand how their offering benefits them and what it is they do in less than 3 seconds of reading. 

Does your copy resonate? Test it to find out

When visitors land on your site, your messaging can either convince them to buy or leave them to bounce.

Copy testing helps measure how well your target audience responds to your messaging, preventing any friction in the buying process. 

Mixed-method feedback models like Wynter’s testing framework allows you to combine qualitative and quantitative insights to verify what doesn't resonate with your customer base and improve your messaging.

Know exactly what your buyers want and improve your messaging

Join 10,000+ other marketers and subscribe and get weekly insights on how to land more customers quicker with a better go-to-market machine.
You subscribed successfully.