In-Ear Insights: Marketing AI Conference 2023 In Review

In this episode of In-Ear Insights, the Trust Insights podcast, Katie and Chris discuss takeaways from the recent Marketing AI Conference, MAICON 2023. We talk about the growth of the event, Chris’s generative AI keynote, and share highlights from other speakers on AI regulation, ethics, and impact on jobs. We debate whether agile development processes are still needed with AI. Katie challenges the idea that QA processes can be fully automated. We speculate on an artisanal future where automation and human craft co-exist. Join us for a lively discussion on where AI is taking marketing and business.

[podcastsponsor]

Watch the video here:

In-Ear Insights: Marketing AI Conference 2023 In Review

Watch this video on YouTube

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher Penn 0:00

In this week’s In-Ear Insights we are back from MAICON, the marketing AI conference, the conference that we’ve been a part of since its very first event.

In fact, Katie, you and I spoke at the 2019, the inaugural event.

And this year, it was bigger and better than ever, with more than doubled the number of participants from the inaugural year, almost more than actually more than triple the number of people from last year.

And from what Paul Rhett’s or the founder was saying, they’ve had so much interest in everything they have to offer that they are bursting at the seams.

Because of generative AI.

Every talk was generative AI.

In fact, traditional machine learning stuff barely even mentioned was all general AI all time, mostly large language models.

I did the opening keynote, which seems to be well received and things but in terms of the themes, and sort of the big picture for where marketers think generative AI is really much I would say most of the crowd was still in the What is this stuff phase? Or how do I use this productively? Or is this going to be this you know, the singularity and Skynet? Sort of those?

Katie Robbert 1:18

Well, can I just first I want to say, you know how proud I am of Paul and Kathy and Mike and the rest of the team, because they have been beating this drum for years trying to get people to pay attention.

Even, I wouldn’t say even more so than you, Chris.

But just in a different like, I mean, they’re putting a whole event together, they’re building their whole company around marketing AI is literally called the marketing AI Institute.

And so to see them have their moment in the sun was just so great, because this is they’ve never wavered from their message.

And the industry finally caught up enough for people to like, oh, well, this is where I need to be.

So I’m just so proud of them.

And so happy for them.

That this is how it turned out like they stayed the course.

And I think there’s you know, without getting too deep, like there’s definitely a lesson there of like, stay true to what you know, and stay true to your message.

And people will catch up eventually.

And for them, they finally did.

So I’m just I’m just so happy for them.

That this is how it worked out.

And so, you know, so you did the opening keynote on large learning models.

And so for those who weren’t there at MAE con that mean, so it wasn’t a breakout session.

Everybody at the event was in one of the main ballrooms, and Chris took the stage and scared the bejesus out of everybody.

But I will say, as you were talking, Chris, on stage, I was back here in Massachusetts, getting text messages from a variety of people about how much you were killing it.

So do you want to talk a little bit about what the main purpose of that talk was? Like? What were you educating people on? Because the large learning models wasn’t just about generative AI?

Christopher Penn 3:11

Well, so I talked about generative AI, most of the focus was on large language models, you know, everything from GPT-4, to Claude to to, you know, take your pick of of all these different things.

So is an introduction, what is this stuff? How do these things, how to companies make these things and then we spent a lot of time on use cases, like practical, hey, go, you know, take this slide, and go try this in the system of your choice, whether it’s ChatGPT, or Claude or whoever, go try the things so you can see it for yourself.

And then we talked a bit about the risks, and then spent some time on where this stuff is going from, from our perspective.

And one of the biggest things that I’ve really been been hammering on for the last two or three months, is how, how open source adds a big, big, big wrench in the works in terms of the spread of this technology.

The reason for that is when it was just OpenAI and maybe Google and Microsoft, right? There’s some big names, big vendors.

Cool.

Everybody understands that.

And then meta comes along, says, Hey, here’s this wonderful model that we’re sure will everybody who have a model for you and Amalfi they did the Oprah thing.

And this had a couple of major major impacts one, it just took a baseball bat to the knees of regulation because you can’t regulate it anymore.

You can’t say OpenAI, Microsoft and Google this is the these are the rules that was just gonna happen.

Facebook’s Like everybody can have it now.

Good luck, which I thought was so clever because it basically just shut down regulation before could begin while Congress is discussing like, hey, what do we do about this? How do we regulate OpenAI? Facebook’s Like, well now, good luck regulating this suckers.

Katie Robbert 5:01

Ah, Facebook.

What? Okay, so it’s a hot mess.

We know this.

What were some of the other sessions or talks that stood out to us because one of the things that I know events are struggling with is they want to get AI experts.

But for a lot of people AI is still so new that it’s really hard to find true experts who’ve been doing this for a while, because you can have you can stick anyone up on stage and say, This is how you build a product, whether it’s good or not, the audience may not know.

So what were some of the other sessions, from true AI experts that stuck out to you that were really valuable that people should be following and paying attention to.

Christopher Penn 5:48

So one was actually it was AI, but not in terms of AI expertise.

But in terms of legal expertise, our friend and law firm, Shannon toric, toric law took the stage on Thursday to talk about AI regulation, copyright and all that stuff.

And her session was really informative, because one of the things I didn’t realize was, you can have copyrights, and trademarks and stuff like that service marks are things that protects your intellectual property.

But trade secrets, are not don’t have the same kind of protection, because they are inherently secret.

If they are disclosed, they’re no longer secret.

And they you no longer have claimed to them, but you can’t lay claim to them without converting them into like a patent or a trademark or a copyright.

So she was saying, if you put trade secret information into a prompt that then goes into a large language model like GPT-4, you have forfeited your right to call that a trade secret because you handed it over to a third party.

I’m like, Oh, shit, right? If I think about all the people who are handling, handling data and going, let’s put it into this into ChatGPT.

Like, that’s, that’s super dangerous.

Katie Robbert 6:56

What if you build your own private large learning model? Like I guess so.

And that’s sort of the question and I know that you’ve talked about this a lot is, there’s the public, large learning model, which everybody is using and asking really bizarre questions, which is probably why it’s getting dumber, or there’s building your own.

So if you build your own, and host your own, I would assume that it’s then okay to share trade secrets and even, you know, protected, personally identifiable information, if it’s your own personal large learning model? Is that a correct assumption?

Christopher Penn 7:34

That’s correct.

So if you were to use a tool, for example, like cobalt, or llama CPP, or probably the one that’s easiest for people to use, would be this one that’s called LM Studio, you download the the model of your choice, right.

And I have like 14 different models.

So if Karen, the editor, for example, and you run this locally, this runs on your laptop, you need like a decent gaming laptops, but this runs on your laptop, you can unplug all your internet cables, turn off your Wi Fi, and this will work fine, because it’s not sending data out anywhere.

It’s not recording any data.

And therefore you are protecting your secrets, your your your confidential information, and stuff like that.

So that is that would be an example to your point of Yeah, you could you could do that with those things.

One other thing I didn’t know from recession is the IP address has migrated from a PII to, to sensitive protected information.

And in a healthcare context, Amanda toda vich, from Cleveland Clinic, so this, the AI your IP address is in a healthcare context is considered protected health information like, oh, because Google Analytics logs that she’s like, Yeah, we can’t use Google Analytics, we are forbidden to use Google Analytics because it logs IPs.

And that is protected health information.

Now I’m like,

Katie Robbert 8:54

Okay, that’s a big deal.

Yeah, I there’s a lot to unpack there.

And I’m gonna write that down for a different episode, because there’s a lot to unpack there.

Christopher Penn 9:08

But the short of that is, if you were putting like server logs to do some anomaly detection, and you were using a third party tool, like ChatGPT You can’t do that anymore.

You are violating HIPAA.

If you use

Katie Robbert 9:23

Yeah, that’s that’s a BFT.

Exactly mildly what were some of Did you you know, without, you know, throwing anyone under the bus.

But did you come across any sessions where the information was just flat out wrong? That was being shared? I’m

Christopher Penn 9:45

not wrong and sensitive, like technically wrong.

There were some sections which I thought had a maybe a bias towards a certain point of view that I don’t necessarily personally agree with doesn’t make it wrong, but It’s not how I would approach say, a given issue.

And then there were some sessions that were just mind blowing.

So Professor Ethan Malik’s closing day one keynote, he talked about where AI is going.

And he’s he’s a really, really well educated thought out person in the space.

And he was like, Yeah, but what is happening right now in AI is sort of the height of irresponsibility, in terms of the the large scale implications for these different tools and technologies to the point where what big tech companies are doing isn’t is hazardous is his point of view, you’re handing tools out to people that are like handing chainsaws to toddlers, and then you’re wondering why people are getting hurt.

Because no one no one is, by handing out access to models that are essentially raw materials, you’re putting the onus of responsibility for their their intelligent use to people who don’t know how to use them.

So again, it’s like handing chainsaws to people have never had a chainsaw, and then just hoping they figure it out.

And not providing more than the minimal amount of guidance.

But he said a couple of other things that were really interesting, one of which I wanted to tell you about.

But he did say, in terms of tools, what tools to use, stick to the foundation models, which is something we say all the time, like, yeah, there’s a gazillion have vendors, but you should learn how ChatGPT itself works, you should use him.

He said if you want to survive as a practitioner in your field, is one of his four rules, use AI for everything, use AI for things that even you don’t even think you should use AI for is used for everything so that you understand implicitly what it is and is not good at and how fast it’s evolving.

Because you want to go use a large language model for something last week, and it couldn’t do it this week.

It can he’s like, that’s the only way you’ll know how things are evolving is by using this.

But the other thing he said was that development, software development is the most impacted area by AI above and beyond anything else, he said 70% of ChatGPT usage is software development now.

And he said, This is the part I want to get your take on agile development is no longer necessary, because you no longer need code review cycles, you can just hand off to machine to do QA.

Katie Robbert 12:25

I get it from a efficiency standpoint.

But you’re sort of back in that developers reviewing their own code challenge.

And so I do disagree with that.

I think at some point, this, that will be a viable thing.

But from my viewpoint, and of course, I haven’t dug into any of this yet, I’m just hearing it for the first time.

But my off the cuff reaction is you have to have two different, you know, systems stood up.

So like you can’t have ChatGPT reviewing ChatGPT, you would want to have some other large learning model reviewing and sort of a gut check and a counterbalance against what was sort of this is the reason why QA engineers exist in the first place.

You know, so you would want to have, you know, the code check system, separate out, you know, standing separately from, you know, whatever it is you’re doing in order for that to be a true statement.

So I do see a risk, at least where, you know, AI is in its infancy in terms of accuracy and reliability, you know, it’s not accurate 100% of the time.

And that, to me is problematic, to make the statement that you won’t need QA engineers, I do feel that you will still need that human intervention.

Because the machine knows what humans know.

And if the software developer is making errors, that is introducing those errors into the machine.

And that’s then what the machine knows.

And that to me is where you still need someone separate and independent from that.

And so, you know, it’s going to learn over time, it’s going to get smarter, it’s going to be less error prone.

But it’s still never going to be a perfect system.

Christopher Penn 14:27

Do you agree with his argument that agile itself is unnecessary because you don’t need the two weeks scrums anymore, you can you and ChatGPT or you Anthropos clog, can iterate so much faster and out of that cycle of you know, stand up scrum to do two weeks.

You can shorten that down to multiple cycles within a day.

Katie Robbert 14:50

I think it’s true in terms of the actual development work getting done.

I think that you probably don’t need you know the Agile to week it or RIF cycles from the actual work getting done standpoint, but from a planning standpoint, I still think that, you know, some kind of planning, whether it’s two weeks or two days or two hours, there still needs to be some kind of, Hey, what did we do? What is the outcome? What are we expecting this thing? You know, because the challenges that you’re, he’s talking about black boxing everything down into, okay, one person sitting in front of a computer just doing things.

And then the rest of the product team or the company going, Well, what happened? Well, it’s fine.

The large learning model handled it right.

But what happened? What changed? What did we do? What are we telling our customers? What are the release notes? Where do we have to look for possible bugs down the line, because there’s a ripple effect of these things.

So agile or any kind of planning, you don’t even have to call outs, you know, just be smart about it, you still want to do requirements, you still want to have outcomes, you still want to have measurement, like those things, you won’t want to skip over.

And those are those foundational pieces.

You know, maybe the machines move faster than humans, great.

The machines aren’t.

Unless you’re programming in this way, the machines aren’t telling you, here’s what the customer asked for.

And here’s when we can deliver it.

So you still need that, you still need all of the pieces around the actual development work happening.

Christopher Penn 16:31

Yep.

And then the day to opening keynote was actually the one that I wanted to see.

And for just fortunately, we’ll see, you know, the vast majority of Cassie calls off the Chief Data Science chief decision scientist at Google was talking about what jobs AI will take.

And conceptually, you know, pretty much everyone agrees on on on the same foundational concept that AI can’t take a job.

But it absolutely can take individual tasks.

Because you automate away tasks, she says something very interesting that I think you might have a perspective on as well.

Because you know, me and my tendency to not document things.

You said, the way AI works is it learns by example, right? You, you, we call it data, but it learns by example.

And she said, if you find a task, that it’s easier for you to just do the task than to explain it or documented, that task should go to AI, because the instructions are too complex for you to articulate, but the machine can learn from your examples.

And then so in terms of identifying tasks, for machines to take over her perspective was things that you just don’t want to explain.

That’s something a machine should do.

Katie Robbert 17:41

So the challenge there is that that’s literally everything for you.

And so I can see where I can totally understand her perspective.

Because the machine is essentially, it has to, you know, just in very basic terms, it has to do the documentation for it to be able to do it in a repeated way.

So, you know, we’re not seeing the documentation happening, but the machine has to catalog step by step, too.

So when you say, hey, machine, do this thing, the machines like, let me go back into my library of things, find that process, and then execute it.

And so I can totally see that perspective.

What I would want to see is then can you retrieve that process from the machine in order to create training materials for humans, more other machines? Right? And then how do you validate that the steps were correct? So, you know, again, sort of going into that black box of software development? Sure, you can train the machine to do the thing.

But how do you know that the thing is correct.

Christopher Penn 18:52

The other thing I thought was interesting, is that she said in terms of what tasks machines will take over versus what tasks humans will still remain uniquely human.

She said, Humans inherently are creative.

We are creators, we are thinkers.

And we are.

And that’s the part that, you know, we enjoy and we’re good at where’s the doing part, the execution part? That is simply typically something machines are better at.

And so if you said, hey, I want to take jobs, it will just take away all the tasks that are execution based that you would have doing the thing.

And I sat there and thought to myself, if you were trying to put lipstick on the pig of you know, who’s going to lose their jobs, that’s not a very thick coat of lipstick, because essentially, you just said that at like, 80% of your job is doing stuff, right.

Which means that if that’s the part that machine is going to do, the 80% of the tasks in every job are going to be handed off to a machine.

What are you gonna do with all the extra people because that’s, I mean, that’s tremendous amount of of labor that’s being offered will be offloaded to machinery.

And I agree But it, it was contradictory kind of to me, to me, it was contradictory to the message, you know, AI is not gonna take your job Oh, yeah, it’s gonna take 80% of your job.

And, and not.

And I say this nicely.

Not every job requires a lot of thinking and innovation, right? The person who cleans my hotel room does not have to enter visit.

In fact, they’re explicitly told not to innovate, like, just do the thing.

You know, the sheets go this way, the furniture should not be glued to the ceiling, things like that if the person who the person who hands me the stuff at McDonald’s after I order it, again, innovation is not really a feature of that system.

In fact, innovation is strongly discouraged, follow the procedures.

And we’ve seen, you know, if you go to McDonald’s, and a good portion of each individual task is now automated, like when you watch the put fries, and there’s a machine that measures up the number of fries as a machine that measures how long it’s been in the fryer and stuff like that.

So the human is basically just moving a basket around him.

And that’s about it for that job.

It’s not what it used to be.

So I feel like she’s 100% Correct.

But the message was disguised to be something that was more reassuring than it should have been.

Katie Robbert 21:23

I think that, you know, that future state is a few years off.

But you’re you’re right, I think that those repetitive tasks, you know, it’s interesting, I, you know, every once in a while, like you and I will talk about this, like, Oh, I just need some kind of mindless task to just like, you know, it’s like I, you know, you appreciate the monotony of just doing something mindless sometimes.

And you’re absolutely right, those are the tasks that will no longer exist.

I think for some companies, the team members won’t be given a choice, the company will say, this is what we’re going to do, all of these mindless, repetitive tasks are not gonna be done by machines.

I think there will be some hold out companies, probably smaller companies that won’t have artificial intelligence doing those things.

So you know, I sort of think, and I hate hate the term artisanal.

But I sort of see the industry will split.

And you’ll have the AI.

And then you’ll have the artisanal companies where things are done by hand.

I mean, if you look at, you know, so Amazon versus Etsy, for example, you know, you have Amazon, who’s basically the big box store, who has run a lot of people out of jobs.

And then you have Etsy, which is the collection of small businesses, for the most part, where things are handmade.

And, you know, people are looking for more of that human experience.

I think that companies like even big marketing companies, B2B companies, they’re going to you know, it, there’s going to be a divide, you’re going to have the companies are going to, you know, use AI to automate everything, and then you’re going to have the company to like, cool, AI is great.

But if you still want that human handmade thing, that we can still do that.

So I think that there’s going to be that battle, and people are going to start to choose sides of do I want automation? Or do I want or too little?

Christopher Penn 23:33

I think you’re right, because there is that sense.

There is a sense among some folks that because a lot of the machinery is black box, you don’t necessarily know that you can trust it all that much.

Whereas you even though technically your brain is also black box, you having a another human being working on something, yeah, there’s still going to be mistakes in it, but at least you you know who to ask about the mistake, you know, there’s there’s someone that you can hold accountable, whereas, you know, a neural network with 80 layers and four dropouts in a softmax layer at the end, there’s no one you can go to and say, Hey, this thing didn’t do what was supposed to do.

Why Why didn’t it? And so I agree with that.

The other thing that Professor Maalik talks about which talks, which dovetails into this, is that we are at a turning point right now in AI, where we have to decide as this, these these generative systems begin to consume everything.

What does the future look like for work? Do we turn over our tasks as we should write to the to the machines where it’s appropriate, and enjoy the fruits of that, that labor of you know, more free time, more time to think more time to spend with your family more time to take up golfing, you know, whatever? Or does it become a hyper capitalist hellscape where you have 120 employees but you have 15x the profitability and you just continue to work everyone to death.

And he said Those seem to be the two forks in the road the two branches in the forks in the road And he said, it’s not clear which way society will go.

And I’m sitting in the audience trying not to yell course, that’s clear.

That’s just America, it’s a hyper capitalist hellscape you know which road we’re gonna take.

Katie Robbert 25:13

But I think that’s where you go back to, you know, companies making those choices of, you know, fully automated versus, you know, things done by hand by humans.

You know, it.

And I feel like this is a deeper conversation for another episode.

But I do feel like, you know, the way in which we think about, you know, hourly work, and how we bring home salaries, and what the value, you know, of the things that we do our that needs, a lot of examination.

You know, there’s, you know, we sort of poke fun at the thought leaders, people who are just, you know, want to get paid to think and not do anything, but, hey, guess what we just talked about for 30 minutes.

And so, you know, we need to rethink what that business model looks like.

And so there’s a lot of work to do, there’s a lot of work to do.

And I think that we as humans are not keeping up with how fast AI is changing things.

And that’s where, you know, that’s where that fork in the road is, you know, we may find ourselves going down the path of hyper capitalism for now, because we just can’t keep up.

But I do feel like, that doesn’t mean that that is going to be the only road forever, I feel like there’s going to be, you know, side roads and branches off that road of people starting to figure out, okay, this is what a new business model could look like, let me, you know, spin off from this hyper capitalism road and go back the other way.

It’s just gonna take time, and people have to be able to make those adjustments to their businesses.

Christopher Penn 26:55

Yeah, it’s gonna be very interesting how this all plays out.

But so that was, those are some of the big highlights from the event.

I’ve, I found it incredibly refreshing.

It was a wonderful event.

Again, kudos to Paul and Ashley and Kathy and everyone on the team for putting on a fantastic event.

Hopefully, we will be back again next year.

Of course, you know, we don’t want to make presumptions.

But I would encourage anyone who’s thinking about, you want to learn more about marketing AI.

If you’re not already hanging out with us, that’d be a great event to go to.

And if you are hanging out with us, then come find us at that event.

And we can all hang out together and speculate as to what the different speakers are saying.

I’ll leave you with one last funny quote from the closing day two keynote from Olivia Gamblin was an ethicist who took the whole session on AI ethics.

She threw out this little gem says compliance basically means you’re one step above the illegal, like, that’s a great quote.

Katie Robbert 27:52

I can see I’m gonna hear this one a lot.

Christopher Penn 27:56

We actually got we were hanging out the bar afterwards.

And she and I got into a very long philosophical discussion about the nature of fairness and what constitutes fairness from an ethics perspective.

That is a whole other bag of worms that maybe maybe sometime we’ll tackle or MAE, who knows, maybe we’ll even have her on the show or something.

Katie Robbert 28:16

That sounds good.

Well, Chris, you know, I’m, I’m glad that you had such an educational and wonderful time at mekon.

I know you also did a deck, are your slides available for people who may have missed your talk?

Christopher Penn 28:31

Funny that I don’t have the slides available.

But within an hour and a half of getting on stage, the entire talk was available because I am a nerd.

And you can go get that where can I get the slides.com

Katie Robbert 28:43

so easy to remember, it really is.

Christopher Penn 28:45

Everyone had a good laugh at that.

And if you’re not sure about that, or you just want to chat about it.

You can also get it as well as a bunch of other things in our free slack.

Go to trust insights.ai/analytics for marketers, where you have over 3200 other marketers are asking and answering each other’s questions about data, analytics and AI every single day.

And wherever it is you watch or listen to the show.

If there’s a challenge you’d rather have it on.

Go to trust insights.ai/ti podcast, chances are we have it on the platform of your choice.

Thanks for tuning in.

I will talk to you next time.

Need help with your marketing AI and analytics?