In-Ear Insights: Generative AI Limitations in Marketing

In this week’s In-Ear Insights, Chris and Katie discuss generative AI limitations in marketing and why focusing on generative AI might not be the answer to your woes. You’ll learn the best ways to analyze your data to find the true cause of the problem and get actionable steps to turn things around.

Watch the video here:

Watch this video on YouTube

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher Penn 0:00

In this week’s In-Ear Insights, a follow up from last week’s episode on predictive analytics in generative AI, we got a comment on YouTube folks who said, Hi, I thought it was an interesting topic but didn’t get anything from the episode that was actionable to go experiment with, it’d be helpful to analyze an example of how to solve data, sell funnel leakage problems with generative AI, they give an example of a family law firm for traffic’s going up, and and prospects are going up.

But sales qualified leads and retainers has stayed flat.

And obviously, they’re kind of wondering, I’m guessing they’re kind of wondering why some of the funnels moving upwards, some of it isn’t.

So Katie, when we tackle problems like this for paying clients, we obviously spent a lot more time doing requirements gathering much more than you would get in a YouTube common.

If this person were to say, onboard as a client, what would our first steps be?

Katie Robbert 0:52

I would, I would probably run them through the five P’s first and sort of understand like, what is the purpose? Actually, no, I would step back from there probably and say, you know, I would want to undo? Well, gosh, it’s hard, because there’s so many things that I want to know more about.

And so let me see if I can sort of like restructure it.

So yeah, let’s start with the five piece, because that’s always a really good place to start.

And so, you know, what is the purpose? So the purpose sounds like they are trying to understand, you know, to your point, why some of the funnel is moving, and some of the funnel is stagnant.

And so, you know, so the question they want to answer is, how do I get the rest of the funnel to start moving again? The people so you know, are there salespeople involved? Are there content marketers involved? You know, whoever’s involved? What is the process? I think this to me sounds like where we would probably focus a lot of our time.

Because at a high level, if some of your funnel is working, but down funnel, it’s not that to me says, Well, what are you doing in between those steps to, you know, to keep people engaged to get them to learn more about what it is that you do, too? You know, what is how long is your sales cycle? Is that part of it? So, you know, there’s a lot of questions there a platform, you know, it depends, like, let’s take a look at their CRM, let’s take a look at their website and see if we can figure out if there’s any kind of correlation between when people become sales qualified leads, or when they drop off.

And I think the big question is, you know, at what point are they dropping off or staying stagnant? And then the performance is, Did we answer the question? So, you know, that’s at a high level, but within each of those, there’s a lot to unpack.

And for me, the thing that just keeps sticking is what’s happening between consideration and purchase, that people aren’t moving on.

So you know, making sure you know fully what your customer journey is.

So that’s usually generally like awareness, engagement, consideration, purchase at a very high level.

So awareness of people finding out about you for the first time visiting your website, engagement, people sticking around and maybe setting up for a newsletter or downloading content consideration, that’s when they start to raise their hand and go, you know, what, I think this is the right thing for me.

And that purchases when they complete the transaction.

And so I feel like there’s something not working between that consideration and purchase.

And so that’s where I would want to spend the most time is, What content do you have? Are people getting what they need from you, because you’re giving away too much helpful content, and then they’re leaving, or they’re finding out your prices are wrong, and then they’re leaving, like there’s something not working.

And again, this is without, you know, further context.

So if this was a paying client, we would be able to get into these details.

Chris, what would you add to that?

Christopher Penn 3:49

So I wholeheartedly agree the five P’s is the answer to this question.

And typically, so in marketing, the difference between a marketing qualified lead and a sales qualified lead as a marketing qualified lead has expressed interest and intent.

A sales qualified lead is able to buy right so they’ve got budget, they’ve got a timeline and things like that.

So if you’ve got a fracture between MQL and SQL, is the second of the five Ps.

It’s the people the people that you have as marketing qualified leads may not be buyers, right? You may be Katie, now you and I discuss this all the time, I have the almost timely newsletter list, which is a huge list of 300,000 people.

But it’s not the majority of it seems to not be core buyers of Trust Insights.

So when I send out an email and say like, Hey, you know, sign up for the Trust Insights webinar coming up, whatever we get a lot of marketing qualified leads, we get a lot of people who are interested in our services, but they’re not buyers.

They can’t afford to buy right if it’s if you’re a one person realtor, they can’t afford the our full service they can afford our courses and our smaller offerings, but they can’t afford the oh the white glove service packages that are larger clients have to So are MQ ELLs are very high, our ASCII wills are not because the wrong people.

And I suspect very strongly in this example of, you know, they reciting a law firm mail request, consultations, et cetera.

Yeah, I mean, I can request a consultation from a lawyer to and I can have the free intro 15 minute call, and they’re gonna realize very quickly why you are not qualified to buy our services.

And I think this is a really good example of why you need the five P’s to do the requirements gathering, because this is, you know, the person’s like, hey, how does predictive analytics agenda AI help? It doesn’t? This is not a generative AI problem.

This is a process problem.

Right?

Katie Robbert 5:42

And you know, and that’s something I always go back to the example that I gave when I was talking about this at Social Media Marketing World of, you know, people leading with, what is the problem I want to solve with generative AI? And that’s the wrong question to ask, it’s what is the problem I want to solve? Period.

And the example I gave is, when I was talking with the executive director of the shelter that I volunteer at, she was coming to you with questions about what you know.

So I’ve been looking at generative AI.

And I have this big spreadsheet of resources that I want to put on my website.

And it was a mismatch, she needed a directory plugin for her website, which by the way, she did figure it out.

They’re going to do a subdomain, because the resources piece is so big, but generative AI doesn’t play a role in it.

And I think that that’s one of the challenges that we’re facing today.

Is generative AI is dominating the conversation.

It’s dominating the news cycle, there’s so many things that you can do with it.

Is it going to take my job candidate do the thing that we just automatic? We’ve we’ve created this bias that, well, it must be a generative AI solution.

And it it might not be it might be a very, gosh, it sounds so weird to say it might be an old school solution that doesn’t include generative AI, I put that in air quotes, where it’s really just, you know, you have to manually take a look at what you’re doing.

And generative AI doesn’t play a role.

Now, once you figure out what the problem is like, Okay, I have so you could use generative AI to create ideal customer profiles, you could sort of use generative AI to do a high level analysis of who your MQ ELLs are, and who your SQL czar, and see like, what’s the difference? Like? Where does it break down or get a better ideal customer profile of your paying costs customers? And then see, is that who’s coming to my site? Is that who’s getting into my funnel? If the answer is no, that’s your problem.

And then you can use generative AI to help you put together some sort of a marketing plan.

But that analysis of your funnel is not where generative AI belongs.

That’s a process problem.

That’s a people problem.

And those are the two things in the five P’s that solidly do not need generative AI, those need human intervention.

And those are the two things in the five peas that are 100% of the time, the most broken.

Christopher Penn 8:24

And the hardest to solve, and the hardest to solve.

But yeah, so here’s a generative AI suggestive approach that I would recommend.

And we’re going to do this because it mirrors exactly what you were just saying.

Okay, so first, we’re gonna go ahead, and we’re gonna pre prime the model, we’re gonna say, let’s talk about marketing strategy and funnel leakage.

What do you know about this topic, especially in the context of law firms not getting sales qualified leads? Right, so we’re going to start it off by filling essentially having it right its own prompt of here’s what I need to understand about this general topic, right? Here’s reasons for leakage.

Why law firms struggle with this sort of thing.

So already, just by asking the question, you have one of the things I think people forget is that gender to me, I can be a good partner to you.

It can be a good, you know, a second opinion, I think, I think gender AI in all instances is a great conversation partner, because it’s so different than talking to Wilson the volleyball right? Except you will actually get better answers.

So we’ve gotten the basic now let’s take without the mentions of the gender of AI part because again, we’ve we’ve ascertained immediately this is not a gender of a problem.

So we’re gonna go Okay, great.

Here is a current problem our law firm is facing, please assess and analyze it.

And I’m going to put in just here’s what’s going on.

Right, what would you advise

Katie Robbert 9:52

and so as this is generating the response, the thing to remember, and this is what you tell me all the time, Chris is that gender Word of AI still has a very high rate of hallucination.

And so even if you’re asking specific questions, it doesn’t mean that the answers are correct.

We know, which is why you ask upfront, what do you know about this topic.

And even if it says all the right things, it may still not give you good information back, you know, maybe it’s having an off day, maybe it didn’t have coffee that morning, or, you know, maybe it found just the wrong resources to pull from.

So you still need that human intervention.

So we’re not solving the problem with generative AI.

We’re just sort of doing a little bit of a gut check of are these did I think of everything that I can possibly do to solve this problem? It’s essentially, you know, I’ll probably get a lot of hate for this, but it’s eventually Is it a really sophisticated internet search of solutions at this stage at this particular stage?

Christopher Penn 10:57

Let’s take a look at what it came up with misaligned targeting your marketing attracts people who aren’t a good fit match your ICP ineffective qualification your intake process may not be effectively identifying things, value proposition potential clients may not clearly see how you solve their problems.

Why did you choose you know the competition, nurturing neglect, you might be giving up too quickly on leads, economic factors, an actionable advice data, deep dive, looking analysts throw their way or drop offs happen, or the demographics lead to changing even while traffic increases.

That’s a really important one.

In in Google Analytics, in Adobe analytics, and all these services, you can gather some percentage of demographic information.

But also that should be part of the forums on your website.

If you have a fortresses not only how did you hear about us, but Who the heck are you reexamine your ideal client? Again, that’s a case where it’s, it’s a data problem.

It could be a classical AI problem with building a regression analysis for what factors determine a, a, an ideal client versus not, but it’s not gonna be a generative AI problem, right? Do an intake, audit, differentiate nurture, economic agility? So these answers that I came up with, to exactly your point, Katie, these are your next avenues of investigation? How are you getting traffic? Where are you getting a traffic from? Are you getting your traffic from Craigslist ads that, you know, that is still a thing? Oh, they are gonna get a very different quality of traffic than you get from, say, a bespoke publication in your city.

Katie Robbert 12:25

And so, you know, back to the original question of, you know, how can I word, you know, make predictive analytics and generative AI more actionable in this particular case? The answer is, you’re not there yet.

First, you need to understand all of these pieces you need to understand, you know, so using predictive analytics is not going to tell you why you’re having leakage issues.

So one of the things that we talk about is the data analytics hierarchy.

And predictive is maybe the third or fourth step up, whereas most companies are stuck at the first baseline, which is what happened.

So it’s what happened, why did it happen? What can I do about it? And then I know there’s two more but one of them is like, how can I get the machines to do it for me? And so a lot of companies are feeling like they want to be at the top of that hierarchy.

But they’re stuck solely at the bottom, because they still aren’t at the what happens, like they can’t answer those questions.

Easily.

That’s where you have to start is what the heck happened in my data before you can take it to the next level of how can I forecast this? How can I get generative AI to help me through this? And that would be if this were a paying client? That would be the advice I would give is, you know, like, slap their hands away from generative AI? I’m not really maybe depends on the client.

And then I would say, let’s just do an analysis of what’s going on in your system first.

And then once we figure out the what happened, why did it happen, then we can start to bring predictive back end because you want to be using predictive analytics on the right data on the correct data.

Because if you’re doing forecasts on the wrong SQL, or the wrong MQLs, then you’re just going to perpetuate the leakage.

So you know, Chris, let’s say, you know, we use your newsletter, subscriber rates, the month over month, the opens and all that sort of stuff to figure out when our revenues are going to go up.

Well, we already know that it’s your newsletter is great for awareness.

But it’s not great down the funnel.

So we would be predicting on the wrong data and we would be putting resources against the wrong information and continuing to not bring in sales we’d actually be, you know, losing money at that point.

Christopher Penn 14:53

Exactly.

This is a case where predictive analytics with your current data will make things worse.

You’ll make things worse because you We’ll go down those wrong rat holes, right? If you go and you run a predictive forecast on your current marketing channels, and you say, well, emails, the best channel or social media, the best channel? Well, yeah, for the wrong audience.

It’s like, it’s like saying, We’re gonna drive faster on this road, well, you’re on the wrong road, driving faster is not going to help you, you’re still going in the wrong direction, you need to stop the car, look at your map, get on the right road, and then worry about drive how fast you drive, if you drive your you know, you can’t drive fast enough in the wrong direction to get to where you want to go.

Right.

Katie Robbert 15:35

So yeah, I mean, yeah, I’m sure someone will happily debate you on that it won’t be me.

But I think that that’s really what it comes back to is.

First, you need to understand your data, you need to have good data governance, you need to have a good analysis of what’s happening in your data and why it’s happening.

Before you can get to the stage of using predictive forecasting and generative AI.

And I think that that’s, you know, it’s a common misconception, especially with everything that’s happening in the industry right now is that, well, generative AI can probably just fix this for me, I don’t have to think about it anymore.

And that’s just not true.

You, the humans still have to maintain the systems, you still have to do the analysis, before you can bring it to a generative AI tool.

And that’s why in the five P’s platform comes after people and process because, you know, and this is the issue with things like digital transformation, its platform first, maybe people in process, and what we want to change in that narrative and in the implementation is that its platform last don’t even think about the type of platform you’re using to solve the problem, until you understand the problem until you understand who’s involved, and what the processes are.

And then and only then, can you think about the software, the tools.

So if you’re going into the conversation saying, What is the problem I want to solve with generative AI, you’re already setting yourself up for failure.

Yep.

Christopher Penn 17:14

Now to give a little bit of actionable advice to go experiment with, here’s how I would solve this problem.

What I would do is I would go into the customer CRM, and I ask them, here’s your sales want, you know, here’s all of your sales qualified leads, and and the the clients they turned into, I want you to rate the clients good client, bad client 01, right, you know, good client, bad client, whatever, by whatever criteria, you think that is, we take that entire dataset out of their CRM, then you go to a system like generative AI, and you said, we’re gonna write some Python code.

Today, we’re gonna write an auto regressive algorithm that’s going to do this sort of binary classification produce a essentially a variable importance chart that will will demonstrate which variables contribute to the ideal client, which ones do not, that then would say, Okay, now you have to go back to your marketing automation system and implement a screening process for those specific variables to say, Okay, we’re going to get more of our ideal clients by filtering out people sooner, say, like, yeah, you’re, you know, you’re not a, a trucking company, right.

And our ideal customer profile says, your trucking companies are who we should be serving.

And so you can, you would build that screening process out faster.

And your system, again, the part that uses generative AI is essentially writing the code, you’ve got to know the data, you’ve got to do the analysis on the data, you’ve got to know what to ask the system to write the code for.

And then you’ve got to be able to interpret that data.

And of the five Ps, the gender of AI part in that in the platform part is a tiny, little portion.

But that’s the approach that I would take to solve this particular problem.

This is why it’s not something we can demonstrate in a YouTube video, because it would be a YouTube video is like 18 hours long, right, and display a whole bunch of proprietary confidential data that you you definitely do not want people seeing on the internet.

But that’s the approach.

So I would say if this is something that someone wants to go experiment with, that is exactly the approach I would take is do that data governance, do that requirements gathering to find your ideal customer profile, use machine learning, classical machine learning to identify the characteristics of that from the data that you have, then take that finding, test it and implement it in your marketing automation systems.

And hopefully, you get your, you hopefully figure out that yeah, the traffic you’re getting is not the right traffic, and then you change your marketing to say, Okay, well, we need to chase after those people.

And again, you now take this all the state that you have theta back to generative AI, say let’s refine our ideal customer profile, and fix our marketing from there.

You and I just did this not too long ago, we took a whole bunch of data out of our Hubspot instance and stuff and we built a much better ICP, right?

Katie Robbert 19:51

Yeah, no, and I think that, you know, as you’re describing it, and, of course, in the back of my mind, I’m like, Yeah, you could do Do this with machine learning, you don’t need to depends on how big your database is, it depends on how many customers you have.

So, you know, you could if you know your customer base really, really well like the way that we do, you could very easily look at the types of customers you have and say, yeah, these were good customers are these were the wrong fit, they didn’t turn into long term customers, and then take a look at your like, newsletter subscriber list or your community? And just, you know, very quickly, go.

Yeah, I don’t see a lot of overlap there.

So how do I, you know, go back and get more of these types of clients and really focus on to your point, Chris, those characteristics, how you arrive at that analysis, depends on your skill sets depends on your resources.

And so Chris, and I would have proach, getting to the answer differently.

But we would theoretically come up with the same type of information, it’s just a matter of, you know, what you can do.

And if you have a very large customer base, if you have a very large data set, then you would want to bring on a firm like Trust Insights to help you do that analysis to get there faster, especially if you’re finding that you’re not meeting your targets as quickly as you need to.

I mean, this is true, it doesn’t have to be a law firm.

This is for any company who’s like, I don’t know why my revenue is not moving.

I don’t know what’s wrong with my funnel like that’s the kind of analysis that we can do with our deep bench of knowledge with classical machine learning,

Christopher Penn 21:29

you raise a really good point, sparsity is going to be an issue for companies have a smaller data set, we ran into this in the first year Trust Insights, right, we just didn’t have enough data, because we didn’t have enough clients at that point.

And you’ll find that out in when you do the variable importance analysis, because you will get a measure of statistical significance.

And it will tell you this data set you cannot predict from you cannot do in this analysis from because there’s just not enough data.

What that number is depends on all the algorithms you choose.

And again, that’s something you can ask generative AI for, but you probably should know what it’s saying.

When it says, Well, you know, you could use a Rhema, or ARIMA or any of these different algorithms for time series forecasting, you could use gradient boosting or stochastic gradient descent or, you know, even just flat out linear regression or logistic regression for the for the prediction.

If those words don’t mean anything, that general AI is not going to help anything, because it’s going to be a lot of digging around to figure out what that means.

But that’s at the point where you’d figure out, yeah, this is not going to go well.

And at that point, you then absolutely should be relying on essentially human intuition.

Right? When you’re talking about, say, like, if you accompany sir, if you know, as very specific kind of company, and you know, in your region, there’s 10 of them.

If you don’t need AI for that, you need to go in and go to the golf course and spend time with it.

But 10 CEOs of those 10 companies, I would

Katie Robbert 22:50

like to break that stereotype just for a second.

So I’m a CEO of a company, and I hate golf.

So if someone says, Hey, I want to take you to the golf course, I would say you need to keep walking and find someone else.

I hate golf.

But I understand what you’re saying.

So basically, you need to find a way to reach, you know, those decision makers, you know, and so it really depends on Well, I guess it really starts with Do you know who your ideal client is? And are you getting that ideal client? And if the answer is, I think so maybe you need to be 100% sure about that, because that is going to dictate all of the other things that you’re doing.

Exactly,

Christopher Penn 23:24

exactly.

And there is there is no substitute for the legwork.

There is no substitute for the data governance for the research and stuff.

Generative AI helps.

As we’ve said many times, low language models, in particular help with language problems.

The comment that the person left on YouTube is not a language problem.

It is a data quality problem.

And generally, I will not make that better.

And predictive analytics will make that worse.

So if you do have a situation like this, where you have some questions about, you know, here’s our contact information, just go to TrustInsights.ai AI slash contact and we’re happy to have a chat with you about solving the problem and whether you’re solving it with the right tools or not, because that’s literally what we do if you’ve got some scenarios that you want to share and have a chat with.

Go to our free slack group go to trust insights.ai/at analytics for marketers where you have over 3000 other marketers are asking and answering each other’s questions every single day.

And wherever it is you watch or listen to today’s show.

If there’s a challenge you rather have it on instead, go to trust insights.ai/ti podcast, and you can find us on most podcast sources and while you’re on your channel of choice, please leave us a rating and a review.

It does help to share the show.

Thanks for tuning in.

I will talk to you next time.

Need help with your marketing AI and analytics?