In-Ear Insights: MAICON 2024 Key Takeaways

In this episode of In-Ear Insights, the Trust Insights podcast, Katie and Chris discuss their key takeaways from the Marketing AI Conference (MAICON) 2024. Learn why scaling AI beyond consumer tools like ChatGPT is crucial for businesses looking to leverage its full potential. Discover why process decomposition is essential for successfully integrating AI into your workflow and how it allows you to identify specific tasks where AI can truly shine. Understand the importance of establishing clear business requirements and standard operating procedures before diving headfirst into AI implementation. Gain insights into how to evaluate new AI models as they emerge and learn to differentiate the hype from the practical applications.

Watch the video here:

Watch this video on YouTube

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher S. Penn – 00:00
In this week’s In-Ear Insights, we are back from a week on the road at the Marketing AI Conference in Cleveland, where both Katie and I spoke. Katie spoke on managing the people who manage AI, and I spoke on the importance of open models from a data privacy and business continuity perspective. Plus, we got a chance to hear all sorts of stuff from many different speakers. And on the very last day, two hours before the closing keynote, OpenAI released its new model, Zero One, a new reasoning model. So, there’s no shortage of things to talk about. Katie, when you look back at the whirlwind week that was Marketing AI Conference, what were some of the things that stuck out to you?

Katie Robbert – 00:41
Well, first and foremost, I want to thank the team over at MAICON for doing such an amazing job. Chris, it’s no secret that traveling to conferences isn’t my favorite thing. It’s not my favorite part of the job. However, I will say that the team who puts on MAICON, the team at the Marketing AI Institute, it’s done—and I gave them this feedback—it’s done in such a thoughtful way that it’s hard not to feel comfortable, regardless of your level of social anxiety. They really thought through how to make everybody included in the experience in a way that was going to be comfortable for them. So, I just really want to give them a huge shout-out because they did such a fantastic job.

There was no detail that they didn’t consider, and it really showed that they were really excited to do this for everybody. So, one of the things that I got from the conference was that it was a really nice validation, because sometimes we kind of live in our own little bubble. We forget that there’s other people out there doing similar things, that there’s other people out there talking about the stuff that we talk about—not in a competitive way, but in a peer-to-peer way. And it’s hard for us in the day to get out and see all of that.

And so, sitting through a lot of the different sessions, I wouldn’t say that things came as, “Oh, that’s new. I didn’t know that.” But more of a, “Oh, good. We’re all moving in the same direction, putting our own stamp on it, putting our own spin on it.” And for me, that validation that we’re also doing the right things that are recognized in the industry was a really nice bit of information to get. I also got some really good ideas of other things to do. And so, for example, Lisa Adams gave her talk about content and using generative AI.

Everything she talked about, and she showed her matrix of product market fit, and we have—Chris, you talk about your scoring rubrics a lot—and it’s very similar to that. And it was sort of that, “Oh, duh. We’re not using our scoring rubrics to do product market fit, but we could borrow” because she said, “Take this and use it.” So, I’m absolutely going to say, “Okay, I’m going to borrow what you outlined, put our own spin on it, and see what this looks like for us.” So, it was things like that I picked up on that I was like, “Oh, that’s really cool. That’s a great use case for stuff we’re already doing that doesn’t require us to build something new.”

Christopher S. Penn – 03:30
Yeah, I think the thing that most folks benefited from was hearing all the different use cases. We are still exploring the “What can you do with these tools?” because they are so incredibly versatile and flexible.

Katie Robbert – 03:45
What were some of your big takeaways? And I don’t mean from a technical standpoint because I feel like in terms of how the technology works, you’re at the forefront of that. But what were some of the use cases that you got from the sessions that you attended?

Christopher S. Penn – 04:02
Here’s the big thing that stood out to me. This is both technical and business, is that people are running into scaling issues with generative AI. What I mean by that is everyone’s got and is using consumer tools—ChatGPT, Gemini, Claude—and we’re typing in a browser window and stuff like that. “Okay, do the thing. Cool, it looks great.” Those are single-use consumer uses of these tools, and they’re great. There’s nothing wrong with that. “Hey, help me write this blog post, proofread this script,” and so on and so forth. But to scale, you need more than that.

So, in one session, someone asked, “I have 20 petabytes of audio data spanning 20 years. How can I use generative AI to process this?” That is way beyond the scope of ChatGPT. That is at the point where you need infrastructure. The way we often talk about this is that a tool like ChatGPT and the model is basically the engine, and you kind of need the rest of the car. If you’re using the web browser, you’re basically like sitting on the engine trying to manipulate the wheels. You don’t have a seat, there’s no steering wheel, there’s no roof over your head.

And the rest of that infrastructure is more traditional in nature, but the web interface does not provide that and will not provide that because it’s outside the scope of what pure generative AI can do. So, I heard a lot of use cases like, “I’ve got 700,000 PDFs I need to process. I’ve got this. I want to make this available to my employees. How do I do this? I can’t figure it out looking at the ChatGPT window.” And that, to me, speaks to the importance of knowing the technology, what it can and can’t do, and then knowing what you need to build around it. So, a huge part for everyone is coming up with the 5P framework and your business requirements so that you understand this is the part that generative AI can do, and then here’s all the stuff generative AI can’t do.

Katie Robbert – 06:07
I think it’s really fascinating because we’ve been to conferences before where new technology has been rolled out, but never technology like this where it can do so many different things. I think about a CRM or a marketing automation system—it pretty much does exactly what it says it can do, and that’s it. You’re kind of locked into, “Here are the features, here’s what it does. It doesn’t do anything beyond that.” It’s not the kind of software system that you can get creative with. It does its job. It does the one thing.

And I’m seeing—to you, to echo your point—I’m seeing that struggle with generative AI because, while yes, it’s a piece of software, no, you’re not locked into a feature set. You have to decide what it will and won’t do. And you’re right, people are struggling with that because—I mean, we see, we’ve talked about the—the top use case for how people are using it is content generation. And I think the words I said in my talk were, “It’s the least good use of generative AI, and yet is the most used use case of generative AI.”

And so, when you’re seeing all of these other sessions of use cases of how it can be used, I think it was very overwhelming for the attendees at the conference of, “Oh, wait, it does that, too? Well, I need to do that, but how do I get to start to do that? What system do I do?” And so, it starts that whole process all over again of, “Wait, now I have to build that, too?” And I think it can be just very overwhelming for individuals and companies who are trying to figure out how to scale, exactly.

Christopher S. Penn – 08:01
When you think about these tools, in many ways they’re programming environments, their development environments. When you open up ChatGPT, in some ways, it’s no different than opening up Visual Studio or the IDE of your choice for coding. The difference is you’re coding in English or Ukrainian or Japanese instead of C or Python or Java. But you still have to apply the rigor of and the discipline and the requirements gathering and the thoughtfulness of the SDLC to your prompting. When you are prompting, you are coding. You’re just coding in plain language, but you are still coding.

And I think that fundamental fact gets missed by people because a, most people are not programmers, and b, most people—myself included—are not great at requirements gathering. And so, because we don’t think that through as thoroughly as we should, we don’t write code that’s as effective as it could be. And so, you go on LinkedIn, see all 50 killer jacket prompts that will unlock your marketing mastery. And you look at this prompt, it’s like, “That’s a paragraph. That’s like a Hello World in code.” Like, if you want a killer prompt, show me 2,000 lines of code that have logic loops and conditionals and all that stuff, and then I might believe you that it’s an actual killer prompt.

Katie Robbert – 09:24
Well, and I—but even that—so, you use the RACE and the PARE framework to write your prompts to write, essentially, your system instructions. And so, the RACE framework, you can get at TrustInsights.ai/RACE, and then the PARE framework, TrustInsights.ai/PARE. Using those two frameworks together is going to give you a really robust set of system instructions. But even just talking about that, I can see—when I’m talking with our clients or other people—I can see them start to fade out because it’s a lot. It’s more than just saying, “You are a marketing expert. You will write 500 words on best practices of SEO.”

Like that, people could wrap their heads around, people could say, “Okay, I can do that. That’s not that big of a deal.” But when you start to get into writing system instructions, writing code scaling, that’s where—”But I don’t have those skills. I don’t have time for that. I don’t know what that is. Is that my job? Is that what we need?” And it brings up all of those questioning, “Is this where we need to be going with this? Is this the right thing?” Not because—and the thing is, what’s interesting is people know it’s where they need to go, but that’s sort of their version of dragging their feet of, “But if we do it, we’re doing it. We’re in it. We’re committed to it.”

And you’re absolutely right, because they haven’t done their requirements. It feels really uncomfortable, it feels really uncertain. But if they went back and said, “Okay, what is it that we’re actually trying to do?” then it would be like, “Oh, okay, now I know it’s—” It’s funny, it’s the same way I manage my own personal anxiety. If I don’t know all the pieces, then I’m going into a situation really uncomfortable. But if I do my homework and I gather all of the data and I have an understanding of the different scenarios that could happen and what I would do in those scenarios—yes, it’s a lot of work upfront, but it’s just like requirements gathering—I can walk into any situation and feel 100% comfortable because I’ve accounted for everything. And it’s very—it’s the same process. It’s just a matter of doing your homework upfront. And that’s the piece that people will tell you they don’t have time for, and it’s really just a matter of prioritizing, making the time for it. If it’s that important, you will make time to do it.

Christopher S. Penn – 12:01
Exactly. The other thing I heard a lot at the event is that people are struggling with process decomposition. And what I mean by that is we have been using generative AI and even large language models as sort of an umbrella term and an umbrella concept. “Oh, I’m going to use generative AI to do this task.” And people are struggling with breaking down a task into its subcomponents so they can see what AI can and can’t do. Here’s a simple example, the one’s very familiar: “I want to make a killer presentation with generative AI.” Sounds reasonable. It’s a mostly language-based task.

What goes into a presentation? Number one, need a story or a script or a flow. Can you use generative AI for that part? Yes. Number two, you need a PowerPoint-like structure. Can generative AI do that? No, Python can. Python can create PPTX files, the physical containers, but generative AI cannot do that for you. Number three, you need to break your story into logical small segments. Can you do that with generative AI? Yes. Number four, can you take each segment and put it into a PowerPoint slide with generative AI? No, you cannot. Again, Python can do that, but generative AI can’t.

Number five, I need compelling imagery on my slides. Can generative AI do that? Generative AI can make the images. Generative AI cannot put them on slides for you. So, that’s a combination task, has to be decomposed. Make the images is separate from put the images on slides. And then step six, put the text and the images together. Can generative AI do that? No, it cannot. That’s a PowerPoint function. So, this one seemingly, “Oh, just use GenAI to make a killer presentation,” is actually about 12 different tasks, only half of which you can use generative AI for. And so, when I heard people at MAICON talking about, “I want to use generative AI for this or that,” they were struggling conceptually with understanding, “How do I break a task down into subtasks so that I know what AI can and can’t do?”

Katie Robbert – 14:20
And it—I’m going to be a broken record—but it comes back down to process development. It’s something that—and here’s the thing, you can use, ironically, you can use generative AI to help with the process development in order to figure out where generative AI fits into your process. It’s very meta, but you can do it. You can’t skip the step. I mean, I even heard during sort of the second to last session where Paul and Mica put—were sitting on stage at MAICON talking about, “What do we do with all of this information? I’ve been saying it to anyone who will listen to do process development.”

But when I hear bigger names and bigger voices than mine echoing the same sentiment, I’m like, “Yeah, maybe they’ll listen to that person.” I don’t care who they listen to, you just got to do it. It doesn’t matter if you listen to me or someone else that you trust more. You just got to do it. You can’t skip over process development. And so, Mike was sitting on stage saying, “Hey, you have to do your process. You have to outline.” I’m like, “Yeah, listen to him. If you’re not going to listen to me, listen to him, but you got to do it.” And that’s where people get stuck because they’re not clear on how things are working.

We were talking with a few folks at the conference, and they were asking about, “We want to bring AI into the organization to do things more consistently.” And then in the same breath, they’re saying, “But things have gone sideways with our team, and we have three or four people doing it three or four different ways. And, which way is the right way?” But you have to start there. You have to solve that problem first before you introduce new technology.

And I think that—there weren’t sessions that explicitly talked about that, and perhaps there should have been in, like, the—maybe there should have been a “How Do I Do This?” track. I’m sure people would have signed up for that. I mean, maybe that’s something that will have to happen in the coming years. But I think that “How do I do this?” starts with a consistent process, and that’s something that a lot of companies are going to struggle with. I mean, to be fair, we’re a small company, and there’s certain things that we do that perhaps aren’t done the same way every single time, and they’re only done by one person. And so, it makes it really hard to turn it into an automated process, and we’re just a small company. If you think about the companies that have 20, 30, 40, 100, 500 people, there are people in roles where their sole focus is to solve that problem because that’s how important it is.

Christopher S. Penn – 17:23
Exactly. Yeah. When you take a company like IBM with 300,000 employees, at that point, you—almost everything has to be standardized because otherwise, you will just not be able to function. It’s kind of like the US military. The US military has very clear roles and a chain of command and all this stuff for good reason because when you have 1.1 million people who have to all row in the same direction, sometimes literally, you cannot get too creative or you get chaos.

Katie Robbert – 17:57
It’s—I mean, that’s when I—I always say, “New technology doesn’t solve old problems.” That is exactly what I’m speaking to. Slapping generative AI on top of a chaotic work process isn’t going to fix it. No, it’s more interesting chaos.

Christopher S. Penn – 18:15
Chaos.

Katie Robbert – 18:17
Interesting being subjective, but yeah. So, when I think about what was lacking at the event—lacking is sort of a harsh term—but the—I think the missed opportunity was to have one or two sessions that were really, “How do I do this?” And I know that they have workshops, but those aren’t available to everybody who attends the event. That’s sort of, you have to sign up for those specifically. But perhaps, not so much a hands-on, but like, “Let’s take 40 minutes to walk through how to build an SOP” because, to be fair, I don’t think even that is not something that, if you’ve never had to do it, you’re probably like, “Wait, what does that mean? Where do I start?”

Or, “Here are the tools that can assist you in building an SOP,” or whatever it is. So, I think that in terms of what was missing, I think those kinds of things. And it’s tough, you can’t cover everything. But I feel like maybe moving forward, that would be a valuable session that people would sign up for.

Christopher S. Penn – 19:27
Yeah, I think a Fundamentals or even a Prerequisites for AI session or even track. I mean, that’s practically a workshop on its own of, “How do I help people get ready for AI? How do I decompose a process into its subcomponents so that I know this part can be handled by AI, this part cannot right now?” That doesn’t exist. I don’t think I’ve seen that at any conference. What people tend to do is what you warned us about in your talk, which you can download at WhereCanIGetTheSlides.com, which is people put the technology first. “Let me use generative AI for this.”

Well, no, decompose the task first so that you know what can and can’t be done by AI, which, unfortunately, also means you have to know what AI is capable of so that you can say, “Yes, that’s for AI. No, that’s not for AI.”

Katie Robbert – 20:25
It’s interesting, though, because—and this is where I can—I certainly wouldn’t want to be in the position of our friends over at the Marketing AI Institute to have to make those decisions of what sessions go in and what sessions don’t. People show up to the sessions, people show up to these events to get the latest and greatest information. They want to be dazzled. They want to be wowed. They actually go in expecting to have the show—so much overwhelming information. They’re not going there to learn how to build an SOP, and yet that’s what they need.

I was talking with someone—I don’t remember who—but it was basically like, “Everybody wants dessert, but they just need to eat their broccoli first, and then they can get to dessert.” And it’s like, “How do you get them to eat their broccoli so that they know that by the time they get to dessert, it’s worth it?” Whatever the conversation was. And I was like, “That’s exactly it.” Like, unfortunately, I’m broccoli, and Chris gets to be dessert because he’s like, “Here’s the future. Here’s the thing. Here’s the shiny object.” And I’m like, “Yes, but you still need your fiber and your nutrients.” And people just want to skip over that part.

And listen, I’m perfectly fine with that role because it’s important. People still need that piece of it, but they don’t show up to events to get that information, and yet it’s maybe arguably one of the most important takeaways.

Christopher S. Penn – 21:56
Right. Because unless you are already a master chef, you can’t cook without a recipe. I mean, you can, but it’s not going to go well.

Katie Robbert – 22:06
Yeah. So, I think if we get back to the point, overall, I think the MAICON event, the MAICON conference, it was really successful. I think there was a lot of really good takeaways. Everybody’s going to have their own unique set of takeaways. For some people, generative AI was a brand new, shiny object that they really hadn’t played with. For some of us, it’s something that’s built into our day-to-day, and we’re picking up new use cases. And then we all collectively got to see, as you said, the rollout of OpenAI’s Strawberry model. That was a really interesting thing to happen.

I think it was good that it happened at the conference, one, because we all got the information at the same time, and two, you were able to see just how agile that team is and the type of experts that they are that they could pivot that quickly and get the information and deliver it in such a way that was understandable to people of, “And here’s what’s happening.” So, I actually thought that was a really cool moment. I’m sure for them, for Paul specifically, he was probably sweating and like, “Oh my God.” But I think they—I think he handled it really well.

Christopher S. Penn – 23:26
Absolutely. Absolutely. Yeah. It was funny when that came out, I was in the speaker’s room, and a couple of folks just from the conference were hanging out with me. And as soon as it dropped into the—the model card dropped—we should have recorded it. I did not think about it. But the process of evaluating, “Okay, what does this thing do? How does it work? What is under the hood?” And we came up with some very quick conclusions, quick enough that within 20 minutes, I had a video up about it.

We’ll cover the model in more depth probably another time, maybe in a newsletter, something. In fact, maybe I’ll do it in this week’s Trust Insights newsletter, “What to Know About the New Model,” because it’s very different. It does some things not so well that the regular ones do pretty well. But I think there would be value in people seeing, “What’s the process of even evaluating a model as a standard operating procedure?” When a new model drops, you download the system card. You read the system card. If you don’t want to read the system card, you put it in generative AI, have it summarize it to you, figure out what it can and can’t do.

If you get access to it, you have a benchmark prompt. You say, “This is my way of saying I want—I have this one prompt I have that I use all the time for benchmarking a model.” And it’s a very difficult prompt. It’s essentially writing song lyrics and having evaluate itself on it. And so, you can test every model and say, “Okay, how did it do on this task?” And again, I think, like you said, it was really cool that it dropped at MAICON, but there’s no—people don’t have an SOP for evaluating a model.

Katie Robbert – 25:09
Yeah, I guess so. That would be my big takeaway is, “Build your SOPs.” Oh, wait, that’s my takeaway all the time. I don’t even have to go to a conference to tell people that. But, so again, going, refocusing back on the MAICON conference, I think that people learned a lot about what’s possible. And I think that was sort of the goal, was the team who put on the event really wanted to show people what’s possible, get them excited, get them talking about it, get them trying and experimenting. And I would say that they definitely accomplished that goal.

Christopher S. Penn – 25:43
I would 100% agree. And our own takeaway for ourselves, for Trust Insights and our customers, is now that people have a sense of what’s possible now, how do you get there? And our role and our job is to help people go from, “That’s really cool,” to, “Oh, that’s how you do it.” Or maybe this—whatever shiny object isn’t the right tool for the choice, which—shameless plug—if you want some help with that, go to TrustInsights.ai/contact, and we are more than happy to help out with that.

If you have some thoughts about all of the announcements happening in AI recently, or you are at MAICON and you want to contribute your perspective, go to our free Slack group. Go to TrustInsights.ai/analyticsfor marketers, where you and over 3,500 other marketers are asking and answering each other’s questions every single day about data and analytics and data science and AI. And wherever it is you watch or listen to the show, if there’s a channel you’d rather have it on instead, go to TrustInsights.ai/TIpodcast, where you can find us in most places that podcasts are served. Thanks for tuning in, and we’ll talk to you on the next one.

Need help with your marketing AI and analytics?

Machine-Generated Transcript

Leave a Reply Cancel reply

Subscribe to our Weekly Newsletter

Pin It on Pinterest