In-Ear Insights: How To Use Generative AI For Competitive Analysis

In this episode of In-Ear Insights, the Trust Insights podcast, Katie and Chris discuss how large language models with perfect memory can assist with competitive analysis. They explain how these tools can quickly synthesize vast amounts of public data to provide insights into competitors’ offerings, differentiation, and performance. Katie and Chris note the importance of keeping your own data fresh so tools have current info. They give examples of how to leverage these models, from summarizing competitors’ businesses to analyzing website design and content. Katie and Chris emphasize combining machine learning with human expertise to ask the right questions. They advise using AI for competitive analysis as one more tool, along with classic techniques like regression analysis.

[podcastsponsor]

Watch the video here:

In-Ear Insights: How To Use Generative AI For Competitive Analysis

Watch this video on YouTube

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher Penn 0:00

In this week’s In-Ear Insights, competitive analysis is nothing new, we have been doing competitive analysis since the first stall opened at a bazaar in the Middle East.

And when guys look at the other guy’s booth going, hey, those figures look better than mine.

However, we are in a totally new environment now, where you have massive databases, which are essentially what large language models are with perfect memory, meaning that they know so much about an entire landscape.

It with that perspective, knowing that these things have perfect memory and a much bigger point of view than any one person is going to carry around.

Katie, what do you think about how that changes? Competitive analysis? What do you think about how we should be thinking about these tools as a, as a assistant, to helping us be better at competitive analysis?

Katie Robbert 0:52

So John, and I were talking about this at a very high level on last week’s live stream.

And the thing that struck me was that there is an opportunity to think about how do we include artificial intelligence as a set of data points in the competitive analysis.

And so you know, so you have to step back first and figure out where artificial intelligence works in your organization.

And then what it’s actually doing.

And so, you know, for example, if we’re at a very high level, you know, let’s say Trust Insights is generating 90% of their content, using generative AI, does our main competitor want to be ahead of us in terms of how much generative AI content they’re creating, but not just how much they’re creating, but how well it’s performing.

And so there’s different things to think about in terms of where AI fits in.

In terms of using these tools? I think that to start with, it’s good to get sort of that high level summary, one of the things that’s always been a struggle for me in terms of doing the internet research is really trying to figure out in summarize what another company does, because we all like to put the marketing jargon in a website.

And sometimes what we do isn’t super clear, you know, we’ve been guilty of that before.

So I can see using these tools to start to build out those portfolios of your major competitors.

What do they do? How many employees do they have? What have their earnings call, say, but summarizing it all in one place, versus trying to find it from six or seven different outdated sources?

Christopher Penn 2:36

I think that’s really important.

Because when you think about it, they’re synthesis tools, right? They can connect the dots in ways that we can’t, because, again, they have perfect memory of massive MIT databases.

And so one of the things that people struggle with competitive analysis is saying, Well, what is the unique selling proposition of a company and if you were to distill down, say, a company’s LinkedIn posts and a company’s website content, or at least the the major page content, you could potentially get a distilled competitive statement that would articulate their unique selling proposition and then you would perform the same exercise for yourself and say, Well, how do these two things compare? Like are? Are we differentiated? And then also ask these tools? Well, given these two things, A and B, how could we differentiate ourselves more? Because again, part of the heart of competitive analysis is differentiation?

Katie Robbert 3:37

Well, and I feel like first that comes from knowing your brand and your business.

And so what would you say Chris, our the Trust Insights, differentiator is

Christopher Penn 3:49

the main differentiator is that we, we solve problems with a combination of a human perspective, namely yours, and a whole bunch of machines and in which is very much my perspective.

But we put these tools together in ways that other people don’t know how to do.

Because it’s kind of like I always go to cooking analogies, like Yeah, and a lot of people can make pizza.

A lot of people can put together some dough and stuff like that.

But not a lot of people, for example, can can if they had to repair their blender.

And so if you’re trying to make the tomato sauce of the pizza, and you can’t fix a broken one, you got to go out and buy tomato sauce at the store.

And now your pizza is no different than anyone else’s, because you’re buying the same stuff out of a job everyone else’s.

Whereas for a company like Trust Insights, like yeah, we can fix that.

We can augment it, we can change it.

We can find using the five p framework where user stories, the things that would differentiate our clients, and then help them build the people, the processes, and especially the platforms that will accentuate those differentiators in a way that off the shelf software, and maybe off the shelf consulting won’t do.

Katie Robbert 5:11

And I think that’s interesting.

I like that analogy, because that really helps explain.

Because yeah, at our core, we make pizza, all of our competitors make pizza.

It’s how we approach the making of the pizza, how hands on we are versus like, well, we bought a frozen pizza, now we’re going to resell it to you like at a markup price.

Here’s your frozen pizza.

Ours is, you know, we’ve made the dough by hand, we have our 20 year old starter that we use, we’ve made, we’ve made the sauce by hand.

And so here’s all the machines that we used and the ingredients and how we’ve tweaked it over time.

And so you’re getting something customized to you versus what everybody else is getting off the shelf.

And so, you know, when we think about using, so tell me more about, you know, this perfect memory from a machine learning model, because that’s something that’s a new phrase for me.

You know, as of this recording, I haven’t heard it described that way.

So tell me more about what that actually means.

For someone who isn’t in the in the large language model every single day.

Christopher Penn 6:21

Perfect memory means that the large models, particularly very large models, like a GPT-4, for example, or a llama 70 B, or A clogged to have so much information in them.

They’re there.

They’re these massive libraries of probabilities.

And when you have near certain probabilities, they’re effectively our memories.

So for example, by say, I pledge allegiance to the what’s the next word,

Katie Robbert 6:51

Barracuda flag.

Christopher Penn 6:57

The more the more probable something is the more it functions like a memory, right? So if you if you remember, I know your average american reflexively can’t help but say a flag and a lot of circumstances, right? If I say God save depending on your age and generation, you can say God Save the Queen and Godzilla King, most people still say God Save the Queen.

Again, just reflexive just it just happens because it’s a memory.

And that memory is based on statistical probability.

We’ve seen it so many times, you’ve heard it so many times that we remember like that.

machine learning models, especially large language models, have the same recall ability, right? If I say, tell me what you know about trust insights.ai, to Claude to, it’s seen enough of us and heard enough about enough of us that it can recall reasonably accurately who we are.

And we have now expand that to essentially the public Internet, which is what a lot of these these models are trained on.

Because of this, they have the ability to recall with exceptional precision, any of these statistical distributions.

So you know, in the context of competitive analysis, they know us they know our competitors, they know, adjacent companies, they know companies all up and down our supply chain.

And they can recall those details without having to go out and search for them.

Because it’s in the statistical matrix that makes them up.

Yes, sometimes they get wrong hallucinations, right, where they just make things up.

But for the most part, and this has become more and more the case, as these models get better and better, they have that perfect instant recall without having to go out and get more data.

And that’s what this we mean when we talk about perfect memory for a large language model, and why they’re so valuable for competitive analysis.

Because because the tool doesn’t have to go out and gather up stuff, it already knows everything about a space, you can ask very complex questions about competitive analysis, like what’s the what’s the differentiator in the unique selling proposition for this company versus this company with enterprise customers? And it can tie all those individual data points it knows together and give you a coherent answer.

Speaker 2 9:08
Which makes sense.

And I think that that’s incredibly useful use case for these tools.

My question is what happens when the data changes? So for example, you know, God, save the queen versus God save the king.

Like, that’s a change that happened recently.

So how often are these perfect memory models catching up? Conversely, what happens if, you know, Trust Insights, decides, you know, what, we’re going to stop offering data solutions, we’re going to start offering blender solutions, but we haven’t put that on our website.

How does this perfect memory model know that we’ve made that change? It doesn’t, right?

Christopher Penn 9:51

It doesn’t.

It doesn’t right now any more so than our audience would right? If our audience still remembers us as that data company like okay, we need to do a re The brand and you know the same things we do to communicate to people, you have to communicate to machines.

So you’d have to go do press releases have to go be a guest on podcasts, hey, we’re now we’re now a pizza company.

The models themselves change based on the frequency of updates that the model maker chooses to put out there.

So for example, GPT-4 still has not been updated since sep tember 21.

So it’s coming up on two years old right now Claude two is much new in that club two is tuned as of January 2023.

So it has a more or a newer sort of memory.

The the technical way to work around some of that if if freshness is important is to look at a hybrid model system.

So Microsoft Bing, for example, uses GPT-4 as a language model, but it uses it for the language portion, it uses its own database for the data portion.

The same is true for Google Bard.

Google Bard uses Google’s search engine results as the underlying database and then uses its language model to return results.

That’s why the search engine based language interfaces are more accurate, because they’re using a different data store.

And so that from again, from a competitive analysis perspective, if you want, if freshness is important in your industry, you’d want to choose your tools carefully to want to the tools that are that have an established track record, like search engines of keeping up with what’s going on.

Katie Robbert 11:29

But it sounds like the flip side of that is, it’s not enough for you and I to just decide that we’re a pizza company and like, you know, tell our private social community or, you know, start talking about it with our clients that we actually have to take action in a way that the machines can find the information, we need to put it out there.

And I think that that, to me, that’s the key of this particular episode is the machines can help with the competitive analysis.

But if they don’t have the information, then your competitive analysis is no better than anything else you could get from any other tool.

And so if you, your company wants to stay up to date and fresh, and make sure that the machines have the most, you know, correct information, you need to make sure you’re taking those steps.

And it sounds like that’s a little bit different than we used to operate.

Yeah,

Christopher Penn 12:25

that’s correct.

However, you can still use the capabilities of many of the tools, even if you have to provide some of the information yourself, again, tools like Bard, for example, or tools like Claude to can accept a large quantity of information.

And then from that quantity of information they can, you can then have them perform specific tasks.

So a real good example would be UI UX design, to say, like, I want to know about the design of my homepage, right? And me, and I want to know, I want to get a sense of like, is it a good design or not? You don’t necessarily need to have the tool to know what that means in order to have it apply principles and things that it understands.

So you could take a snapshot, a screenshot of your homepage, put it into Google Bard and say, Tell me what you see here, tell me Is this good design or not? You put a snapshot of a competitor’s homepage, say let’s compare A to B, which has the better UI.

And so these are all things that are that you have the tools have perfect memory for the task, even if they don’t have the current information, you just have to provide the current information.

Katie Robbert 13:47

I’m wondering, so you know, when you say, when you ask the tool, like, is this a good design? That feels pretty subjective? Would you need to go deeper? And define what Okay, so that was just for the sake of example, for those listening Chris’s nodding his head, yes, you would need to go deeper to define what good means.

Does it meet, you know, UX standards? Does it meet like the ADA standards? Does it meet, you know, whatever, or compared to my competitors website? Where do they have advantages is are their calls to action easier to find? You know, does their website more clearly defined what they do? You know, are? Can it go so far as to know, I’m guessing? I don’t know.

So the answer is probably it depends.

But, you know, can these tools almost act like a Microsoft clarity or hot jar and tell you, you know, what parts of the website are most popular?

Christopher Penn 14:50

I think you would need assistance, explaining what to look for it and, you know, image analysis is something that’s still very, very, very fuzzy but it is possible to at least ask the question and see what it returns.

So for example, here is Google Bard.

I’m gonna go ahead and bring up Google Bard here.

And I fed it a good chunk of the Trust Insights homepage, I had to black out the photos of people because Bart will reject anything that contains human face.

But I can say, I want to know, you know user experience, examine the attached screenshot of a corporate homepage and provide your analysis of its UI your experience and make recommendations to increase both retention and conversion.

It says the homepage is visually appealing uses clear and concise color scheme does a good job of highlighting the benefits of Trust Insights Academy’s program, the call to action buttons are clear and concise, but that can be made more prominent, for example, the button could be larger or have a different color.

Add some more testimonials from satisfied customers offer free trial when the programs create a blog or newsletter, obviously, you know, there’s these are things that you would refine the prompt or have additional conversations like yeah, that it’s already on there.

The fact that it did not spot our blog means that it’s not prominent enough in the image, it didn’t know that there was a blog there.

So even in this very simple, simplistic example, we can see that it has knowledge of design and UX.

It has knowledge of basic principles, we gave it very specific parameters.

We want recommendations to increase retention and conversion, which is what we define it with success, not just good design, but does it help convert, and then it gave decent responses.

Katie Robbert 16:24

Interesting well, and so I can see this sort of opening the door for not just competitive analysis, but just your own QA.

For those who don’t know, QA stands for Quality Assurance.

And so basically your quality control quality assurance, which is a best practice for any project, not just a software project, basically someone double checking your work, making sure you’ve done your homework correctly.

And you’ve used the process that has been outlined and you’re getting the expected results.

And so I could see saving a lot of time and headache, where you’re kind of guessing and hoping for the best when you create a new design, marking it up and saying, Hey, what are we missing here? So you could even feed the system, your mock ups? Before you implement them to say, like, you know, what, what am I missing here?

Christopher Penn 17:20

You answer because you’re and this applies to.

It’s called style transfer.

This applies to any kind of competitive analysis where you’re looking at people’s content to try and figure out what is about that content.

That’s good.

And can I replicate it.

So let me show you another example.

This is a piece of software, it’s a Python package called matching.

What matching does in short, is it takes a piece of audio and this I’ll put up a screenshot of this, it takes a piece of audio, and it’s so this is a one piece.

And then here is a reference piece, right, which is a different song.

And then here is it takes the your source piece looks at the reference piece and says okay, I’m gonna apply the same style of mastering to this audio to make it sound like the reference piece.

So I can now if I liked the sound of hey, I like how Taylor Swift song sounds, I can copy the style of it without but with my own music, my own instruments, my own compositions.

So it’s not copying the physical notes.

It’s just copying how it how it sounds in the year.

That’s an example of style transfer.

If you were to, if you were a Twitch streamer, or any live stream or any other podcast, or maybe you saw something on NPS fresh air, I could take a screenshot of you, Katie right now.

And I could take a screenshot of that, say, critique Katie’s lighting design, what things could I do to make it look more like gar fresh air stage to make it more visually appealing? So again, with it’s it’s all competitive analysis, so looking at existing examples and saying, here’s a, here’s b, how could a become more like B if we know that b is what we want? As long as we as the subject matter experts understand that lighting is a differentiator or sound.

A sound style is a differentiator, then we know what to ask for the hard part for marketers, and everybody is having the subject matter expertise to know what to ask for.

Katie Robbert 19:18

Well, and this sort of this strikes me as really important, because it’s sort of the just because you could doesn’t mean you should, you know, so I think a lot of us, you know, especially if we’re just, you know, if we’re launching new services, if we’re launching a new company, or if we’re just sort of like trying to shake things up, we look at our competitors and go, Oh, I liked the look of that.

Let me replicate it.

But we don’t stop to consider is that the right move for us? And so using these tools to do that analysis of your competitors website of their services of their content of their pages to see where they are not necessarily getting it right And then bringing that information back to your own team to go.

Okay, so we really liked the look of Chris Penn website.

But when we ran it through our tools, we found out that there were like, three major, you know, Ada violations.

And you know, there was no call to action buttons.

So, before we go ahead and overhaul our website to look exactly like his, let’s make sure we’re taking these things into consideration.

And so I can definitely see where that is insanely useful for a competitive analysis.

Christopher Penn 20:32

Yes, and one of the things that we’ve kind of forgotten in the rush towards using generative AI for everything is that there’s a lot of hidden factors, right, that classical machine learning, like regression analysis might reveal.

So let’s say you wanted to be the most popular person on Instagram, you could download all the data and all the images and all the things of top Instagram influencers, when we’ve done this in the past and run a regression analysis on the visible factors, like, you know, number of likes, number of comments, certain topics and things, the regression has always come on pretty weak, it’s always come out like on a zero to one scale, where one is this is this is the thing, and zero is doesn’t matter.

Most of the time, those factors come at like a point to 2.3 not statistically significant to barely statistically significant.

And that hints at the fact that there’s stuff happening behind the scenes that we can’t see.

But you know, in your own company, there’s a lot of things that happen behind closed doors that can influence your success.

You know, for example, we have our colleague, John, we, every time we talk about Trust, Insights, marketing, we’re not saying anything about what John’s doing behind closed doors, nothing illegal,

Katie Robbert 21:39

I think, unethical, oh, no, just normal business development stuff.

Christopher Penn 21:43

But he’s really good at a specific style.

And you can’t see that from our public data.

Katie Robbert 21:50

And that’s, that goes back to my question about that perfect memory of its perfect on what it knows, right.

But if it doesn’t know that you’ve changed direction, or you’ve launched a new service and just haven’t put it on your website, then it doesn’t matter.

Christopher Penn 22:06

Exactly.

Right.

Exactly.

Right.

So yeah, they, they are good at understanding disciplines, they are good at understanding specific tasks you want to ask about, but they don’t know what they can’t see just like a human right, there’s no way we can know what’s going on behind closed doors at Apple Computer, right? But you can only know what what they’ve talked about or where there are leaks along the supply chain.

So but I would strongly encourage people to look at these tools as competitive analysis tools in your toolkit.

So there’s still a lot to be said for the classics, right? You know, doing regression analysis, gathering data from API’s and things there’s a ton to be said for that.

The large language model is one more tool in the toolkit.

Katie Robbert 22:52

I have a lot of ideas of how to use this.

So this is this is exciting for me.

Christopher Penn 22:57

It is exciting and it is if it’s exciting for you and you want to talk about ask questions, go to trust insights.ai/analytics For markers are free slack group with over 3300 marketers who all carefully keeping an eye on each other for competitive best practices.

Now, there’s asking answer each other’s questions every single day.

And if you’ve got a platform, you would rather have our show on this go to trust insights.ai/ti podcast where you can find us on almost every channel.

Thanks for tuning in, and we’ll talk to you next time.

Need help with your marketing AI and analytics?