How to Measure Creative Ideas

Scroll Top

June 30, 2022 Brand Leadership Business Growth Career Growth Creative Agency Design Leadership Mentorship Strategy

Make a difference with data-driven creative work.

When your creativity is helping to drive a business, you need to go with more than just your gut to convince leadership about your campaign ideas. Although measuring creativity can be tricky, the data it provides is what gives creative leaders the firepower necessary to greenlight new projects and make them a success.

In this episode of Real Creative Leadership, Adam shares his secrets for measuring creativity before it goes to market — ones that give you information you can count on. He also offers his expertise in helping leadership think through primary motivations so creative teams can get to quality ideas faster.

Join us to discover how to make creative decisions using the one-two punch of data and emotion.

Watch this episode to learn:

How to use the trade-off triangle with decision-makers
Why measuring emotion — not logic — is what it’s all about
Where data is most helpful in the creative campaign process
What we get wrong by focusing on time spent instead of output

mentioned in this show:

Adam Morgan

Creative Leader and RCL Host

The Stoke Group

Digital Marketing and Full-Service Content Agency

Transcript

Adam Morgan:

Hello and welcome to Real Creative Leadership. Today I do not have a guest appearance on the show today. Instead, we’re going to do a solo episode where I get to dig deeper into a specific topic. And today’s topic is, How Do You Measure Creative Ideas? So this is a really interesting one, and I get this question pretty often when I’m out talking with groups, which is “Okay, great. We believe in creativity, we believe it’s important.” And normally later on in the process of most businesses, there are ways to measure how creative is doing, right? We can measure metrics on opens or signups or content that has form forms you have to fill out, or we can follow AQL or Tans, or MQL, whatever you do to follow your content along your pipeline.

We can always do that when it’s in market. So we have plenty of ways to see how people are connecting. But what we don’t know is before something goes out in the world, how do we know if it’s good? How do we know if it’s right? Up until now, most people just use their gut of like, “Oh, this feels like a good campaign or a good creative idea.” But there are so many out there in today’s business that just don’t trust the gut in hoping that it’s going to be alright. They want a little more data on it. So really we’re talking about how do you measure creativity, but it’s really this idea of data driven creativity. How do we get to it before it goes out into the real world? How do we have data on stuff that we know it’s going to work or it’s going to do well? So that’s what we’re going to talk a little bit about today.

So let’s talk about how we actually measure creativity today. So there are four big categories I’m going to talk about. First is focus groups, online surveys is number two, and then we do expert interviews, and finally user testing. And of those four methods, the interesting thing — and I’ve seen this my whole career since the mid 90s — it’s like, okay, we’ve got these ideas, we’ve got to go out and test, we’ve got to go out and find out what are these ways that we’re testing like focus groups and online surveys, when you really stop and think about it? What you’re measuring is a creative idea in a very uncreative format.

For example, for a focus group, when you put people in a room and you pay them 50 bucks to give us their opinion on whatever your creative idea is, they are not in a normal, natural state of mind. They’re in a state of mind that if they pay me 50 bucks, I gotta be very logical, and I gotta solve their problems and think about this in a way that helps the client. And they’re not thinking about themselves or how they feel or what their experience is. They’re getting jumbled up with all the other things that are going on.

And the same thing is true with online surveys, even though it sounds like, oh, it’s in the quiet of their own personal time, they can decide. But it’s almost like a school test. It’s really logic. You’re testing ABCD, whatever it is, you’re asking them to logically respond to very emotional experiences. And even in an expert interview, it really depends on the person doing the interview. If they can set them at ease and make them start to just share their experience and their feelings, but even still, when you’re doing user testing on interviews, people are putting on that special, I-need-to-solve-these-problems hat. They’re not just responding to stuff as if it were in the wild.

So when you get results, you may get different results in the wild when people are actually reacting to your creative ideas, or they’re reacting to your campaigns. So there’s a problem with that. And I won’t go into that much into more detail about that because there are plenty of books on why focus groups fail and online surveys are flawed, and we’re going to get into that.

And the first big question we want to ask before we even get into all of it is going back to what I call the trade off triangle. And the trade off triangle is this idea that I’ve learned early on in my career and it’s held pretty true where it says, there are three different things you’re going to get out of the creative. You can either try and get it done fast, you can get it done cheap, or you can have quality. And of those three options, you can’t have all three. Every business thinks that they’re going to get all three, but you just can’t. So you’re going to have to have trade offs.

Sometimes you can have two, you can either have it fast in some decent quality, or you can have it cheap and fast, or good quality and maybe you can get that from someone who will give it to you at an affordable price. But the reality is you really should just pick one because that’s what the focus should be. So the question is, before we measure, what is it that you want from your creative ideas or your creative team? Do you want just work that’s fast and gets out in market quickly? Do you want it to be cheap and affordable? Is that your biggest thing that you’re worried about?

And you’re trying to get metrics for your company. “Okay, how do I prove that my creative team is affordable, or I need to get more money so I can bring on another agency or do whatever I need to do.” What is your primary motivation? I think you should start with that one. And because the reality is, sometimes we let fast and cheap cloud our ideas, and the reality of what we’re really trying to measure is, is this a quality idea? So that’s what I’m going to focus on for most of this: how do we get to quality ideas? Not necessarily, how do we get them faster or cheaper. That’s a completely different thing. And you just need to understand that. And then as you’re going into how you’re valuing creative ideas, you’re not clouding in those two other ideas.

So to start, and the reason why when I talked about all those other different measurement methods and why it’s flawed is because at the heart of it, creativity is emotion. Experiences, customer experiences, are really about emotional experiences. There’s a great quote from Brian Solis, where he talked about “what is an experience” in his book, The X. And what he said is, “An experience is an emotional connection to a moment.” And I couldn’t agree more because really my job, as a creative person for two and a half decades, it’s really been about how do I simplify ideas down to their emotional core, and how do I connect emotionally with this audience? How do I empathize with this audience? How do I understand what they’re going through in a very human way? And then show them ideas or share ideas with them that are very emotional. That’s really what it’s all about.

And if you think again about all the ways we are measuring, it really goes back to this whole idea of logic versus emotion. Most of our measurement systems are all around logic, multiple choice, survey, quizzing people, or interviewing them and asking the questions. And we’re really logically trying to ask them to answer our questions. Now, there are some companies who are out there moving along and doing some great different things where they’re trying to understand the two different systems of the brain to try and get out an emotional response. We had one great example at an Adobe Summit several years ago, where Rana June and her company came, and they’d hook someone up to this machine and show them an ad, or show them some creative, and they could measure exactly what’s going on emotionally, rather than people logically switching to their conscious brain and responding answers. It just happened naturally.

And there are some other really good strides that are happening. I know there are a couple companies and we’ll leave it in the notes here, of companies that are using your Apple Watch. And they’re able to measure a different heart rate and other things, as people are experiencing creative ads and ideas, and they’re able to give you a good response as to how people are reacting to it emotionally. So those are strides in the right direction. And I know there are other research companies like BrainJuicer or Millward Brown, who are also trying to work on that emotional system and try and get research that hits on the emotions rather than always logic. So there are good strides happening, which is hopeful.

And really from my journey of understanding and thinking about this whole logic and emotion, it was really when I was working on my book seven or eight years ago really understanding the value of emotion in how people react to ideas versus the value of logic. And the problem is we’re always trying to pit them against each other, where it’s like, oh no, we’re measuring by logic, therefore, it should be built by logic. And so an ad that makes sense, versus an ad that’s emotional, we can’t measure the emotion. And so it’s really convoluted, and it’s really hard. But the truth is we need both. You can’t just have one over the other. So the more we make strides toward understanding and measuring the emotional side of things, I think the better we’ll finally get a good idea of how creative ideas are actually moving forward.

Okay, but until then, what do we do? So I’m going to walk you through a couple examples of just some research or some other things that I’ve learned that can help us have some answers right now on how we can understand a little bit better these emotions and be able to make good decisions based on that before we put something out into the world.

So the first story I want to talk to you about is Paul Zak. So he is the father of neuroeconomics. And what neuroeconomics is, it’s basically the intersection of neuroscience and money, right? And so how do we understand what the brain is doing and how that affects money? And there was a great story from Paul of where he was on a plane, and he was watching Million Dollar Baby. And by the end of the movie, he was just crying his guts out. And at the end, he was like, “If you would’ve asked me to give money towards that cause, I would’ve given you a ton of money.” And that made him think, okay, so how do we get there? What are some of the key indicators that were happening in this movie or the creative that got me to want to just give up a whole bunch of money?

So he went back to his lab, and he ended up figuring out that there are two key ingredients that are important that drive us to action. And those are an anomaly and an emotion. So basically, if something’s unique and different, that makes us pay attention. And then if there’s emotion present, that is where we actually care about it and either make decisions, burn in memories, do all sorts of things. But basically, that’s what leads us to action. So really critical that those two ingredients are super important. You need to have an anomaly, and you need to have an emotion, and therefore you’ll get action. And so the way he did it is in his lab, he could measure attention with things like a quickened heartbeat or sweat from the endocrine glands in the skin. And then for emotion, he ended up measuring the vagus nerve with an electrocardiogram because that was tied to how much oxytocin was being released.

And so he could measure all that and get to an 82% accuracy on any piece of creative and find out if people were going to spend money, right? And that’s really what we want at the end of the day. But if we stop and say, okay, we can’t every single time go to Paul Zak’s lab and hook our customers up and see how they react. I’m sure we could, for some things, if you’ve got the budget, but most of us don’t have the budget to test everything like that. So at least we can try and take those learnings from his little formula, which is: is it an anomaly and is there emotion present? Those two things? Then yes, you have a good chance that people are going to pay you money.

But if you go back and look at how we measure today, focus groups, and online surveys, and all those things — we’re not really able to figure out from those if there’s an anomaly or if there’s emotion because people aren’t necessarily aware of it. That’s what’s been so hard around emotion is if we’re not literally measuring emotion, how are they acting or reacting in a logical way? So that’s what’s been the challenge. First step, just learn Paul’s Zak’s trick.

So if you think about that formula, you need emotion and you need an anomaly. And another way of thinking about creativity is just thinking of creativity as the scientific method, right? It’s so funny that we’re talking about creativity, but it’s going to be a very scientific way of studying it, right? Because the scientific method is really about going out and testing a thing. If it keeps repeating and it keeps happening, then that thing must be true. And so that’s what we’re going to do is say, how do we know if these ads or these campaigns are emotional and if they have anomalies? How do we know if that’s there? And if they are there, then voila, it must be true. So this first idea of data and creativity, how do we get those together? So the easy first test is, all right, how do you know that you’re building engaging content?

The first thing you need to start with are audience insights. And that’s where the logical half of this comes in. That’s where you need to go out and discover what people care about, what they’re thinking about, what matters to them. And that’s really where you can use a lot of your behavioral measurement tools, right? Get some behavioral data where you can see, okay, you can use some of those other tools on your website, in your user studies, whatever it may be, just to really find out what people care about. So don’t have them measure the creative content, just figure out what they care about. And that’s where you want to use logic. So that’s where focus groups and interviews and all those things are really useful. Figure out what I call the human triggers, what they deep down are thinking about and caring about and what matters to them.

Then next, go out and create a creative experience that matches with that. So once you know if a certain audience cares about this one thing, go out and explore, what is something unique about that? What is something newsworthy? What is something different? What is something that’s going to stand out? And then wrap it in emotion, in an idea that connects with that audience. And then finally, the way you measure it is: does A+B=C? Do you have the triggers, and do the creative ideas connect to those triggers? And if they do then voila, then you know you’re going to have some good success. And I know that’s just a cheap and dirty little way of thinking about it, but it’s just good to change your mindset: I don’t have to go out and try and get from some random person on the street using their logic back to me. But I want to know first what they care about and then build the creative, but [I] don’t measure the creative in the focus group. Instead, just make sure it connects with those emotional triggers. That’s all I’m saying.

All right, this next one I think is a little tighter on how you can get data driven creativity. And this comes from a process at VaynerMedia. So I had a friend who worked there. He was their chief creative officer. And this was a system that they used to get real data on creativity before it goes out the door, right? Or at least a cheaper way before you invest in a big, huge campaign. And they used this method, which is a lesson that we can learn from TV pilots, right? So if we think about TV pilots, right, every year we get a whole bunch of new shows that come out, and what they do is they create a pilot for each of these shows. They’ll go out and film an actual real life piece of creative, and then they’ll put them on the stations, put them in market, and see which ones are working, right?

And obviously the ones that do well, the people respond well too, they keep those and build more and more episodes. And the ones that people didn’t like, they’ll stop, right? Good example of this Seinfeld, right? Seinfeld did horrible in focus groups because people were just looking at it logically. They’re like, “What is this? This is nothing like what we’re used to. This is so different.” But when they actually built the pilot of Seinfeld and put it out into the world, suddenly people got it because they were in a natural environment in a real place of viewing it. It wasn’t forced, or an unnatural environment, like a focus group. And they were able just to experience it and emotionally react to it. And then Seinfeld obviously did amazing.

So that’s the principle. What do we do then? How do we apply that to creative ideas? Well, here’s the idea. Just like with pilots, what you need to do is, let’s say you’re about to build a big campaign. There’s already a natural process where the creative team gets a brief, and they go out and figure out a whole bunch of ideas, right? A whole bunch of stories. So that’s the first step, build a whole bunch of little mini campaigns. And maybe you build them in really cheap ways that you can get them out in the world because that’s the trick. If you want to, just like pilots, actually put it out into a real natural environment. Don’t just go off into a focus group, don’t spend a ton of money just studying and researching it in unnatural environments. But instead go out and build a little mini social campaign, do some little social videos or some little social banners or whatever it may be. Social is a great place to test the scientific method.

So you go out there, and let’s say you build four or five little mini campaigns, put them in the real world. And then just see how customers react, just like the TV pilots. And then collect data. Which ones are they clicking on? Which ones are they sharing? Which ones do they care about? Because it’s a very cheap and fast way to just test that, right, without building out the huge, big campaign, putting all the money into it. And then once you figure out which ones people are responding to, which of the campaigns they’re actually working, then put real money behind it.

Now I know that sounds stupid. You’re like, well, Adam, that’s just like putting it out in the world then. It is and it isn’t because it’s really just a test, right? So instead of just… Most people are doing all this research before they even put anything into the world. They’re just going out to a focus group and saying, “Hey, tell us if this is going to work or not.” And then people are obviously going to give bad advice, just like the Seinfeld story. But if you just build little mini campaigns and a whole bunch of them and stick them out there in the world and they’re cheap and fast and just let the creative team go crazy, then you can actually get real data on all of those in the natural environment where people are. And they’ll tell you the absolute truth, and they’re going to give you an emotional reaction, not a logical reaction.

So that’s really a way of getting an emotional reaction to a brand new piece of creative. And then just go crazy, then you start to put that in market, right? So the way that works is it’s really all about knowing the audience behaviors. Like I said before, that little formula, get your logical data to figure out what they care about. Make a whole bunch of stuff, go out and test it in a cheap and fast way, and then whatever’s working make more of it. I mean, that’s really the real data on the creative. It’s the cheapest way to get real data on something you’re about to put in the world. And go all in on the best creative. I mean, that’s really the easy answer, right? That’s how we should test creative ideas. Make it real, and then just test it quickly.

But instead, what are other ways that we’re trying to measure, whether we’re successful or our creative team is successful? And now I’m going to switch gears a little bit away from the work to more of the team. And another way that I’ve seen people measure creative or creative people, is going back and saying, okay, the way that we can measure if something is successful is, again, let’s go back to that trade off triangle, right? Cheap, fast, or quality. And what they do is we go back and say, “Okay, let’s just measure how efficient the team is. And that’ll be a key indicator of whether our creative team is worth it or not.” And they’ll go back and say, “Okay, let’s have the creative team fill out timesheets, and then we’ll take those time sheets.” And that will indicate if the team is operating in an inefficient way.

And again, if you go back to the trade off triangle, the reason why that doesn’t work is because you’re measuring fast, you’re measuring speed only and not quality. And so instantly from the top, you’re just setting yourself up to fail because you’re just measuring the wrong thing. Instead of measuring the input of time, you should measure the output of the content. So I know that’s not necessarily about putting it out into the world, but it is the step at the very beginning of making sure that you’re setting your team up to succeed by not measuring their time. And I have another article on medium, we can dig deep into that in another way. I’ll put a link to it in the show notes. But it really should be about, okay, let’s measure the outputs, let’s measure what the content is, how much quality it is, and then you know how it’s working by putting it out into this world.

And the problem that a lot of times happens with putting value on content in today’s society is that we start with the wrong focus, and we measure creative in the wrong way. We’re measuring it by time. Again, like I said, how long does it take to make it? Or length? How long the headline is or how fast it is to consume? We keep thinking that because we have limited focus from our audience, everything has to be built really fast and quickly, right? It needs to be social, it needs to be short headlines, everything needs to be above the fold on a webpage, people don’t watch things over two minutes long. And so we need to stop focusing on those things, which are the speed and the time thing, and focus on the quality aspect of that trade off triangle and really think about experiences that are unique and that are special and emotionally charged.

Again, go back to Paul Zak’s formula. Don’t measure time, length and speed. Instead, measure quality and focus on quality. And how do we know if it’s quality? By checking to see if it’s got emotion and anomalies, and that’s easy enough to do or secondarily, put a bunch of pilots out in the world and see how it does.

Okay. Now one more quick thought that I thought was really cool in terms of attention and how we’re measuring all this stuff is from Jay Acunzo. So he had a great podcast and he gave us a lot of great content. I’m going to put a link to his show here in a minute. But he always talked about that when you’re really wanting to measure the value of a creative idea, the thing that you should be measuring is the community and the loyalty around it, right?

Which means instead of just measuring that one content and then trying to put all the focus on just that little piece of content, in his world he says, “It’s all about holding attention, not getting attention.” Maybe years ago it was all about doing something bright and shiny and crazy and trying to get attention. But now the most important value you can put on your creative is, is it building a community and is it building loyalty? And you’ll know that by how big your list is, how much people tune in and care, how much people are engaging. That’s the stuff you should measure. So it’s not even about the creative idea itself. If you’re not getting the community, then change the creative idea and try new pilots. But that’s the thing that you should focus on is building that community and loyalty.

All right. So just some of the key takeaways, I know it felt like I was rambling on a lot of stuff today, but here’s some of the three key takeaways that you should have as we’re measuring creative ideas or creative teams. What should we do to go about doing that? Number one, key takeaway: anomaly and emotion. Remember Paul Zak’s story. If you want to make great content, before you’re even measuring it, make sure you start by creating the right stuff. Does it have anomalies, is it unique, and does it have emotion? If it has those two things, you’re 82% of the way there.

Next in order to get real data on your creative, because again, we’re not measuring end of funnel sales. That’s obviously the best measurement, but sometimes that’s too late. If you want early on to figure out if creative is working, build a bunch of pilots of volume ideas, put them in a real environment so that we’re getting an emotional reaction to it, and then measure the data on those pilots.

And the reason why that’s so great again, is because you could have the CEO of your company, your CMO, they’re just like, “Oh, we hate those ideas. Those are terrible.” But you’re not necessarily the audience. Put them in the real world and see what your audience emotionally responds to, and then you’ll know if you have a winning idea or not. Again, if we want to be data driven about it, put all of our ideas aside, put your leadership’s ideas aside, and just go test a bunch of cheap stuff in social media and see how people react. Then you’ll know if it’s right.

And then last advice, or the last two, first, remember what Jay Acunzo said, start measuring loyalty and communities rather than always just the creative. Put it on the outputs, which leads me to, if you want to measure the value of your creative team, measure their outputs, not their inputs. How many great pieces of video or how many webpages — think about the stuff they’re creating and compare that apples to apples rather than how much time or how fast they’re building it.

All right. Well, that’s it for today’s session. I hope you at least have a little bit of a better idea of at least a different way of thinking about how to measure ideas. As always, you can find us@realcreativeleadership.com or on our YouTube channel or on any of our podcast channels. We would love it if you could engage. Let us know of any topics that really matter to you. I’d love to know from this audience what struggles or problems or things you’re trying to tackle with your creative teams, and then we can build some episodes around that and help give some guidance on it.

We also want to throw a shout out to the Stoke Group who helps produce the show. You can go to the stokegroup.com to learn more about them. They’re a full service agency that can help you build great content and help you measure in both a logical and an emotional way. And again, reach out to us, share, like, do whatever you need to help us grow this community. And again, I would love to build content that is valuable. I get a lot of feedback from many of you, of specific shows that have been super helpful, and I appreciate that. Thank you for reaching out and continue to do so. You can also find me on LinkedIn and let me know your thoughts on the show. So thanks for listening. We’ll see you next time.

Leave a comment Cancel reply

Transcript