How to measure creative effectiveness, with science (not guesswork)

Have you ever spent weeks or months toiling over a marketing campaign, just to see it fall flat with customers when it goes live?

“What went wrong?” your team asks themselves, disheartened (and maybe a little tipsy at the office Happy Hour).

You might’ve missed measuring how attention-grabbing your marketing would be in the real world, and figuring out if customers even noticed your message or visuals.

What makes a piece of marketing or a customer experience attention-grabbing?

It’s down to a psychological principle called Salience.

What is Salience?

Salience describes how prominent or visually striking something is. If an element seems to jump out from its environment, it’s salient. If it blends into the background and takes a while to find, it’s not.

Figuring out if a piece of marketing is salient is really about answering these two questions:

1. Do people notice our creative?
2. Are they noticing the right things?

How Can We Test for Salience?

Salience measurement tools usually take the form of some kind of algorithm that lays a heatmap across an image, like this:

There are lots of salience measurement platforms out there. The one I trust the most is called DragonflyAI (a highly-regarded tool used by clients like Coca-Cola and Microsoft).

I use with my Choice Hacking customer experience clients, in lots of different ways. Here are a few:

• Measure whether a piece of creative is working hard enough to grab attention, or if it needs to be tweaked. Customer attention is a rare commodity, but by using an agile, science-based tool, we can increase the odds that a piece of creative gets noticed (while the campaign is still in development).
• Ensure that during key moments in a customer journey, users notice and can find the “right” things. Whether that’s a CTA or a key piece of information, using an objective tool to look through your customers’ eyes can help you figure out if it’s attention-grabbing.
• Diagnose why creative might be underperforming. Often it’s good creative, but it’s just not noticeable. Or the right messages and images are there, but they’re not easy to find.
• Make the case for simpler creative, or why certain creative elements need to be emphasized through color, luminosity, placement, or size. Most creatives love this example because their instincts are usually watered down by clients. Salience analysis helps make a data-driven case as to why simpler, more focused creative is more effective.
• Help clients who are more data and science-driven understand what makes “good creative.” Obviously, educating clients about what is “good creative” is more than understanding if the creative is salient. A salience measurement tool won’t test for strong concepts or compelling copywriting. But it will help more analytical clients understand if the right information is being seen.
• Help clients make creative choices based on science, not opinion. If your boss says they like the blue option better than the red, and the red was your choice, a salience tool can give unbiased feedback. That means fewer rounds of review and fewer opinions from people outside (and inside) the marketing department.
• Help sell-in creative to clients and their internal stakeholders. Decision-making inside of big companies is often slow, involves too many people, and often involves folks who have opinions about marketing without any skills (legal, compliance, CEOs, etc.). Using a data-based tool like DragonflyAI can help you make an objective case about why a piece of creative works.

Wait, so is this eye-tracking or…?

While the visuals produced by salience tools can look like what eye-tracking tools create, they measure salience - not eye movement. Many clients also use eye-tracking for digital experiences, and each of these measurements can be helpful in different ways.

3 Examples of How Salience Analysis Can Help You

Below I’ve shared three different ways I use salience analysis, but these are by no means the only use cases.

You can measure salience in videos as well as still images, and the applications for creative effectiveness, customer experience, information architecture, and user experience design are nearly limitless.

1. Craft stronger creative, faster.

In this example of a poster from Mcdonald's Australia, the simplicity of the creative makes it more effective.

The original creative:

The most salient spots in this creative are the “wi-fry” signal and the McDonald’s logo — but part of the reason they’re so successful is because they’re surrounded by negative space. Our eyes naturally land on the images because we’re not distracted by extra information.

Now in my experience, most clients would see this poster and think it’s a too simple — a wasted opportunity for copy or additional messaging. Without an emphasis on salience the creative might end up looking something like this (created by me):

You can see how there’s lots more salient stuff, but now the logo and the wi-fry signal aren’t as salient. Most people don’t take the time to read copy anyway, so the entire poster becomes a bit of a waste.

2. Create a simpler, easier customer experience.

In this example from Walmart, you can see that their pickup tower (an automated service where customers can collect their online orders in-store) is incredibly salient.

The original experience:

The salience analysis (via DragonflyAI):

In this case, the sheer size of the pickup tower helps make it salient, but interestingly the most salient bit is actually the screen that says “pickup” — a win-win that makes this customer experience simpler and easier for guests.

3. Compare your work against competitors.

In these OOH examples from Coors Light and Budweiser, we can begin to see what’s working (or not working) for each brand. We can use this analysis to spot strategic opportunities to beat out the competition.

The original in-situ creative:

The salience analysis (via DragonflyAI):

On both of these billboards, the copy is doing most of the work. But we should ask ourselves — is that creative strategy appropriate for a billboard that people have fractions of a second to read?

If you were the brand manager for Coors, you might see this as an opportunity for you to stand out. Removing the copy would potentially make your bottle most noticeable which would result in an ad that’s more brand-building than Budweiser’s.

Salient Creative: The Bottom Line

There’s no doubt that salience makes for stronger creative and customer experiences, and in my career I’ve seen millions of dollars in return from strategies that use salience as a key pillar.

That’s why at Choice Hacking, we use behavioral science, psychology, and AI salience tools to ensure our clients’ customer experiences are attention-grabbing, effective, and easy for customers to understand (without sacrificing creativity).