So What? Generative AI tools bakeoff

So What? Marketing Analytics and Insights Live

airs every Thursday at 1 pm EST.

You can watch on YouTube Live. Be sure to subscribe and follow so you never miss an episode!

In this week’s episode of So What? we focus on generative AI tools bakeoff. We walk through which tools generate the best podcast transcript, which tools generate the best podcast graphics and how to set up your own generative AI tools bakeoff based on your needs. Catch the replay here:

So What? Generative AI tools bakeoff

Watch this video on YouTube

In this episode you’ll learn:

Which tools generate the best podcast transcripts
Which tools generate the best podcast graphics
How to set up your own generative AI tools bakeoff based on your needs

Upcoming Episodes:

Have a question or topic you’d like to see us cover? Reach out here: https://www.trustinsights.ai/resources/so-what-the-marketing-analytics-and-insights-show/

AI-Generated Transcript:

Katie Robbert 0:00
The music was hitting me just right today John

John Wall 0:19
Zoolander is totally

Katie Robbert 0:24
welcome to so what the marketing analytics and insights live show I’m Katie joined by John Chris is on an airplane right now. So he’s left us to our own devices to hit the buttons. No, I’m actually really excited about this, John, because I feel like you put together a really interesting research project. And so on today’s episode, we’re covering a generative AI tools Bake Off specifically for podcasts, transcripts and image generation. So we’re gonna go through Jon’s research, he piloted three tools, both for transcripts, and for image generation, and which ones kind of came out on top. But also, you know, if you want to do your own head to head Bake Off, you know, John will walk through his process so that you can set this up for yourself. So, John, before we get into the tools, and you know, all the research that you found, I want to talk a little bit about sort of your needs for the podcast, and sort of what made you choose the tools specifically, so if someone’s trying to do this for themselves, you know, what kind of user story would they need to put together so that they could say, okay, these are the things because it may not be transcripts, it may be something completely different. Maybe it’s editing, maybe it’s, you know, audio quality or, you know, finding guests. So, how did you land on transcripts and graphics?

John Wall 1:49
Yeah, that’s a great question. Because there’s so much going on with AI. And, you know, ChatGPT is kind of dominating the headlines, but there’s millions of other applications. And so for podcasting, the two that are big are transcripts so as to follow the recipe as a podcast Producer, I want to transcribe my audio files, so that the biggest benefit of that is SEO points, you know, the fact that if I have a transcript of the show on the website, now Google and other social networks will be able to see that content up there. And yeah, it’s become kind of the secret weapon of podcasting, like the marketing over coffee domain has been up for 15 years. So it has a domain score of over 70. Like when people sponsor the podcast, just the fact that they get their link on our blog, automatically guarantees that it’s not only going to get a high value domain at marketing over coffee, but it’ll also get on Spotify and all these other domains. And yeah, I had kind of underrated that I didn’t really think about that, as part of the value pitch, you know, we think of it more as a direct marketing play when you make an offer. But the reality is, you can instantly increase your SEO scores and how you’re doing and how your domains treated just by, you know, being part of this site. So transcripts are a huge benefit. But of course, you know, as you well know, it’s like, you either send it to somebody in some, you know, who knows where, where, and they charge you by the minute, and you get it back, and you still have to edit it, or there’s these tools, and that’s what we’re gonna go through, we’ll go through a bunch of them. And then the other one is image generation, which is not unique to podcasting. But, you know, it’s definitely part of podcasting, where you need to come up with a bunch of graphics for any content that you’re creating. And it’s kind of a headache, because it’s, you know, do you do original art, do you do some kind of photo service, and then there’s a whole slew of other issues of, okay, it’s got to be all these different sizes, like this social network over here only takes a square, but that one only takes a rectangle. So we need all these different flavors, and you have to get them all straight and put it together. It’s basically a full time graphic design job, really. So yeah, for that one, it’s you know, as podcasts producer, I want to get graphics that work with the podcast across all social channels, and make the show look professional. Like there’s nothing worse than you’ve got a great graphic, but the left and right sides are chopped off because you’ve got a wrote, you know, rectangle, and it’s a square display on whatever the network is. So that’s what drove me to dive into this basket of tools and see what kind of trouble we can create.

Katie Robbert 4:17
Well, and editing, whether it’s transcript editing, podcast editing, image editing, like that’s a skill set unto itself. And so to your point, that can be a full time job. So if you can find tools that at least get you 90% of the way there I mean, that’s a huge time saver, especially if you know in your case this podcast, it goes out regularly people expected it’s been out for 15 years and there’s an expectation of when it’s going to land regardless of what’s going on in your life. And I don’t you know, I think people at this point now realize that like you’re doing it a to z like start to finish, conceptual booking the gas recording the show editing, producing, posting it from Loading it. And that’s a lot of work. So I’m really interested to see what tools you found. And you know, which ones are sort of coming out on top. So let’s get into it.

John Wall 5:11
All right, yeah, here, I’ll throw this out here, we’ve actually got a list of what we dug into this set the groundwork for everybody. As far as generating images, there were three different tools that were tested Stable Diffusion, dolly to and mid journey. So those are three different platforms that can generate images that we went through. And then for transcripts, Otter, AI, descript, and Adobe Premiere. And then just for fun, we’ve talked about, you know, taking a transcript, and then running it through ChatGPT, to see if it can clean it up more, or get it more accurate. And so we went ahead and did that we’ll have the results for that also. And so to set the stage here for what’s going on, this was an interview I did with Simon Sinek, just over two years ago, that’s up on the site that doesn’t have a transcript because I’m so far behind the transcript generation process. And, but as I dug in further, actually, I think this was an update to what I had done. So I think the original ones up there, but it doesn’t matter, you know, we’ve proven before that this kind of content needs to be updated regularly to get the best SEO benefit from it. So. So again, AI for podcast, image generation is where we started off. And this is what we’ve got for the show, this is the asset pack that I had started with that picture of Simon on the top right, that was from his PR team, that’s the kind of like the go to image that they would just throw you. Over on the left is a marketing over coffee logo we use in a lot of places. And that’s great, because it’s a square, so that works on a lot of platforms. And then I did want to include a portrait a full on pictures, you can see Simon and say, okay, he’s a, he’s a handsome guy, he’s just what you would expect from you know, put a pin in that for when we get into image generation. And we’ll, we’ll see how that goes. Got it. So Stable Diffusion was the first tool that I picked out. So and right out of the gate, you know, part of this was just like digging in and seeing what the user experience was like, and what kind of mistakes you could make. And this was the first one that you know, Damn these marketing people. The site is actually stability.ai. But when I went searching for it, Stable Diffusion dot web actually outranked it. So I went there

Katie Robbert 7:18
first. And of course, I would have made absolutely

John Wall 7:22
everybody would do this. I mean, you’d think accompany would come up with a URL that you know, would lock in and you couldn’t hit. And we see this over and over again. I went down all these weird dead ends. But you know, of course, classic, you know, spammy style. The first thing that happened when I got there is all of these ads are all over the place. But it does still work. And so this is fantastic. The prompt that I use was pod podcast image for the marketing over coffee interview with Simon Sinek. And this is what I got back. Wow. Which Yeah, the one on the top right. I totally love this. That was like, what if Simon was half Popeye or something? It’s just like a completely bonkers photo.

Katie Robbert 8:04
I mean, but in terms of generation, those are kind of cool.

John Wall 8:07
They are. They’re interesting. And so the big theme with this, that came out with an image generation is that artistically, it does an interesting job. But the way these images are generated creates some problems, you know, because when we want to create an image for something, we have a list of requirements were like, okay, the text needs to be this, I need a picture like this, you know, it needs to be laid out in this specific way. But that’s not the way these machines generate images, the, as far as generating the image, it’s actually just saying, Okay, we’re going to start to generate something. And it needs to have as much in common with everything else that we’ve scanned. And that’s similar. So the insane thing is actually these faces are composites of Simon. Plus other guests that have previously been on the show different. Isn’t that bizarre, and it doesn’t do text or export actual text, like you’d look, the writing doesn’t really make sense. Like it, you can kind of see on some of the marketing over coffee. And you can kind of see Simon Sinek over there. But that’s just because other images are sort of like this. And, and you start to see some weirdness to this is something that I saw across the board is you look at this bottom, right, this hand on this guy here is like a little bit too long and weird. And you’ll see the watch too, because they’re composites right? It’s like, okay, marketing logos have watches in them. So let me come together one from, you know, five other images. But so you get all this weirdness. And, you know, the takeaway from that is if you’re doing some kind of article, like a wired feature where you need something a little abstract with a lot of weird random stuff in it, that’s where this stuff can’t shine and come up with some interesting images we’ve actually used in past shows, Chris, did you know a Pokemon Charmander having coffee at a table in this style of a French master you And, you know, and that it can do, it can come up with interesting stuff. But so as I actually, you know, got over to stability a, I got into their tool dream studio, which is it’s a web based tool, you know, you can just jump in there and start creating images. And I did the same prom podcast image for the marketing over coffee podcast interview with Simon Sinek. And it came back with this set here. Which, you know, obviously, the results here are way better as far as just something that you might be able to use a couple of funny things, though. So the coffee cup here on that the number one position has a latte art, you’ll find that pretty much every AI generated the coffee has a latte art, that is just a, you know, that just happens all the time. And then like, I don’t know, at the bottom right there, Simon is like half David Cudney. He’s like, kind of X Files there. And that shot is a little funky. And again, you see the weirdness of this one over at the seven o’clock position. Like you can’t do your hand that way.

Katie Robbert 11:00
I was just looking. I was like, is he is it like another hand coming in? Then he’s like, holding the like, I would drop a coffee cup if I kind of clawed it like that. And then didn’t notice this coming up.

John Wall 11:14
Right, right. And then the other one you have is that thing that sort of like a hat or a speaker or a thing over on the right shoulder there? Yeah, that’s another thing that we’ll see more of these. This is a recurring theme. Also, these are just weird things that come up. But also interesting

Katie Robbert 11:30
how casual they made his like unbuttoned shirt.

John Wall 11:35
Yeah, right. Right, right. And the jacket, like so. Okay, so marketing means you have a jacket. And you know, that’s the way that goes. And a big thing with you, I didn’t really do an objective test, and that I quickly learned that, okay, my prompts suck, you know, I need to be coming up with better prompts, and I’ll get better images. And so that first mistake was that leap, I had to make up like, okay, saying marketing over coffee, and Simon Sinek, these branded nouns is actually not going to get me anywhere, like, you’re not going to get those things back. So like, where do you go? Next step. And actually, this one, when I saw this one, I was like, dude, Davis’s attorney needs to be on this, because as soon as I saw the orange one, I was like, wait a minute, I know that guy. And, you know, obviously, you can see here, it’s yeah, man, those two are very similar there.

Katie Robbert 12:24
So you’re not uploading an image. For reference, it’s you’re using someone whose name is so well known that these systems can find enough versions of him on the internet to generate it. So like, you know, I, I have a web presence, but it’s not as big as someone like Simon Sinek. So if I put in like, you know, headshot photos of Katie robear, it might struggle, if you know, and I can say, with confidence, I know that if you look at my name, and you look at the images, there’s really only three that are repeated over and over again. So it’s going to struggle to create a version of me, who just there’s not as many pictures off.

John Wall 13:04
Right? And there’s a shade of meaning. That’s an excellent question, because we’ll see this again as we go. Because you can in some of these tools, upload images, but it’s only going to flavor the image as far as like, okay, so if that image that you upload is mostly green, it will drag it towards a green tint. You know, it’s it still doesn’t understand that there’s an object in that picture that you want it to be like, or that you want to be at, similar to, it’s solely looking at, like, Okay, this images been composed as far as color and light and what it’s made of, and let me get something close to that. But it’s not going to pull you closer to the subject that you’re actually looking for. So that was a little bit weird. But you know, is how that thing works. Yeah, and so the takeaways were that were just what we said, like forget about proper nouns and brands and work on your prompts. And so this I got to, you know, was kind of the closest step now just a cup of coffee on a studio table next to a Shure microphone. And so coffee. Well, this is a really weird one. And I finally figured out what this is just because I’m a photo nerd. Right there is. Oh, no, a DSLR camera lens coffee mug. Like that’s a thing. Yeah, you’re right. And I had coffee and studio. That’s why we’re getting this weird coffee mug. That’s a camera with a mic cabinet. But and then, much like the hand and the watch thing too. And all these images, like I don’t really know what the heck this thing is. It’s like some kind of coiled cable that doesn’t really plug into anything or go anywhere. Yeah, there’s

Katie Robbert 14:35
only one end. Yeah,

John Wall 14:37
I just, you know, just like, if you’re in a studio, you’re gonna have one of these. And so that was a little bit weird. But these tools all of them have this thing where it usually throws you like three or four. And then you can click through and say okay, no regenerate this one, like this one is closer to where I want to be. So go ahead and you know, throw me for more of these. And so This one was the weirdest of all here same prompt different take. But the mic is actually drinking the coffee. Yeah, I have a lot of questions. That is very weird. And then there’s this like, steam punk weird creeper thing watching me that it’s supposed to be a microphone, I presume. But I like

Katie Robbert 15:19
those old timey dive helmets. Yeah, like,

John Wall 15:23
oh my god, if you play Bioshock, you know, you know that somebody’s coming to get you

Katie Robbert 15:27
like, and so the coffee is on top of something that’s plugged in like a warmer or some sort of device or

John Wall 15:34
some kind of just another device. So you know, generate, again, do another round of that. And here he goes, same problem. And this one was like, hey, that’s actually, you know, that’s straight up. And you know, you could just throw it in Photoshop, throw a logo over there. And that’s like, it’s useful that can be that can work. That’s not bad. Yeah, that’s, you know, we finally managed to kind of get where we could go. And so just to give you the ballpark, it’s 1000 credits for 10 bucks, which is around 5000 images. So cost is not a factor. It’s just like all the others. You know, Chris has always joking about how we get these bills from big data for 35 cents a month, and things like that, you know, it’s that kind of thing you can, you can dig into.

Katie Robbert 16:15
So what’s what’s interesting, and, you know, I can imagine, so toggle, I want to hear you talk a little bit more about uploading the images. Because, you know, for someone like me, who tends to be really camera shy, I don’t like having my picture taken. My headshot is like the one picture that’s out there that’s used for everything, and I’m probably do for new headshots, but the thought of it gives me hives. And so do you think that these, these tools would be good for like, as a, you know, camera shy CEO, I want to use one of these image generation tools to redo my headshots so that I don’t have to sit in front of a live camera.

John Wall 17:00
Yeah, no, I have bad news for you. That’s, you know, that’s kind of one of the things with this is if you have a specific person or brand that you’re shooting for, these tools were like, kind of can’t help you, you, you need to do something else. And even looking at this whole, this whole product segment here, like you still need to know how to use Canva or Photoshop, you know, it’s not to the point where it can do stuff for you. So yeah, even because, again, uploading other images doesn’t get you closer to anything, it just flavors, the output, you know, based on that original thing, but it’s not intelligent enough to know what object you want to flavor on, you know, it considers the background just as important as the picture of your head as everything else in the picture. Yeah, and actually, when we get down to the last one, we’ll show you the use case, you’ll actually see how that works, as far as you know, trying to get close to a person.

Katie Robbert 17:55
And so as you were talking about getting, you know, putting together better prompts. I know that you know, and we’ll probably get into this in the second half. But something that Chris talks about a lot is the prompt engineering. And so he’s put together and you can find this on our website, trust insights.ai/prompts sheet. And it’s a free, you know, no, it’s not gated, you don’t have to give us your information. And basically, it’s just a framework to help you write deeper prompts. But it’s specifically for systems like ChatGPT, I would be interested to know how, how detailed of a prompt you can get on these image generation systems.

John Wall 18:40
Yeah, actually, so that’s one thing that’s really, this is a perfect lead in for talking about mid journey here. Well, let me back up. Oh, all right. No, no, I got to stick up to the program. I don’t lose it here. But mid journey is interesting, because you can see the requests that are that people make and yeah, there’s people that make 10 Paragraph prompts, you know, and the thing they’ll do is they’ll run that huge query and then they’ll edit the text to try and see if they can get it to just go in a little bit different direction to get closer to their vision. And it’s always these far out sci fi crazy stuff usually you know, that they’re trying to get a specific look for but yeah, here let’s jump into it. I want to make sure we plow through all these Delhi to was the next one. And that’s actually on the same platform as ChatGPT. It’s the same OpenAI models, different things. And now this is completely crazy here. I can the same prompt podcast image for marketing over coffee podcast interview with Simon Sinek. But a very narrow band of results here. Yeah. And you know, as you can see here, we got everybody’s got the latte coffee, right. That is a requirement. The other one that got me were wired headphones. And you know, this data is frozen from over a year ago. So that’s understandable that everyone has to have wired earbuds The one that kills me is the last one on the right here though, where the the wire is actually going into a pair of glasses. I’m not sure. Oh, that is

Katie Robbert 20:06
weird works. In the third one what? So you have the coffee that’s floating outside of a cup? And then you have a cup with, like, Kool Aid in it.

John Wall 20:20
Yeah, I think that’s just a logo thing. You know that that’s totally there. And that’s, that’s observant though. We’ll see there’s a couple ones further where there’s some issues with the coffee mug as far as like, Where does the mug begin? And what doesn’t, you know, it’s the same thing.

Katie Robbert 20:36
Like you’d have to write the prompt that says, you know, podcast image remarketing over coffee where the coffee is in the cup.

John Wall 20:43
That wouldn’t hurt. Yeah, I mean, you definitely. And I don’t think it could understand latte. Well, it probably understands latte art. But doing negative latte art, I don’t think is going to help you too much. But yeah, that’s the thing with this. I mean, you can’t just sit down and keep writing and writing and carving and carving and trying to get closer to what you want to dig into. This one was interesting to so and so finally, I got off the proper nouns, again, cup of coffee on a studio table is next to a Shure microphone, the same one that I use for that second batch. But it was just interesting how this was, you know, all very similar, right? The mugs are all about the same, the microphones are pretty much about the same. It’s just different composition of the photos. But and so I’ve decided to go a little bit further down this path to I wanted to see as far as size, you know, can I specify a 25 by 250, but they just all come back a square right? 1024 by 2024. So this doesn’t solve the sizing issue with this tool, at least, like you still need to be able to edit your images and figure out how you want to cut them to get them to work. So but again, if you want an abstract background image for something to throw as a label layer in Photoshop, you know, this can totally do that. And it’s cheaper, easier, faster than stock photography or getting photos done. And then I completely just stole a prompt that someone else had been using going with impressionist oil painting. This is kind of cool, you know, I got something that I kind of liked. And we could just go ahead with something, you know, along those lines. Yeah, it just kind of works. You know, it’s very funny too, in the previous shot, when you look at some of the other options is kind of funny how, you know, you get different types of phones. And actually, we’ll see some of the other phones in the next round that came out even weirder, but you do have to kind of take your eye to it and see what’s in there as far as composition and what goes wrong. Or pricing on that one to 15 bucks for 115 credits. And it’s usually around one credit to generate four images. It’s actually based on processing that it’s like a poor user experience thing. Like you don’t even really know how much it costs. It depends on how much processing time gets chewed up, how difficult it is and how busy they are, it seems like. But again, it’s basically dirt cheap, it doesn’t really cost anything.

Katie Robbert 22:59
I remember when not that long ago, when Dali basically launched their user interface, like the, one of the big topics of conversation is like, oh, you know, I ran out of credits. But you know, my partner gifted me credits, or I got credits from one of my clients. And it’s like, that then became like the new amazon giftcard.

John Wall 23:21
Yeah, the thing that is, and that was the other thing with all these across the board. So I think with the first two, I was able to just run credit cards, and it wasn’t in fact, for all of them, you could run credit cards, but like it was interesting here with mid journey was the next one. They have a quote unquote, free trial. And yet I got you know, quarterbacks that actually, you have to have a subscription because we’re too busy. So you need to go back and pay to get some money to do it, which I did. And now Yeah, mid journey, you’ll be a fan obviously out the gate, because their interface is Discord, you have to have your Discord account set up and you get into Discord. And yeah, if you’re not used to this UI, it is completely overwhelming and crazy. You know, you have the benefit of you see everybody else’s queries going through here. You can see what everybody else is doing.

Katie Robbert 24:13
Which is, which is interesting if you’re trying to learn, but it’s not great. If you’re someone who’s trying to create proprietary images for you know, whatever they use is like that’s tough, because then you’re now in a place where everything’s public. But yeah, this interface, like as cool as it looks is very overwhelming to me.

John Wall 24:34
Yeah, no, this is like Slack. That’s been vomited on. You know,

Katie Robbert 24:39
Chris would be so happy to hear you say that.

John Wall 24:44
It’s, you know, after you’ve been in for three months, you’re like, oh, yeah, this is awesome. But the first three months you’re like, where is everything? You know, like these buttons here. These are all you can upscale an image or you can do a further variant like, well, that’s cool once you know what those are, but until you know what they are, you gotta have to wade through it and figure out what’s going on.

Katie Robbert 25:01
I don’t think I even missed three days.

John Wall 25:04
Just got sick of it. So this is good. Now talking about again, trying to get like if you wanted to get a headshot for an image I started out with, you know, this podcast image. And here’s what it came back with like this first one I love which is like Simon as Phil Collins son. Yes. Random. Isn’t that bizarre? And he’s also

Katie Robbert 25:25
like, it looks like it’s almost like he’s thinking about the cup of coffee so it’s like it should be in a thought bubble.

John Wall 25:32
So and talking about the coffee cups again, I finally realized with this one euro this one is all of these images of latte art. A bunch of them have the hands holding the coffee. So this is actually the hands that are inside the coffee cup they’ve screwed up there isn’t that

Katie Robbert 25:48
and that is very strange.

John Wall 25:51
Yeah, yeah. So that’s so you know these were closer to the mark but again it was that thing of Forget it you know this is you know branded stuff just doesn’t cut it you gotta move on.

Katie Robbert 26:01
Yeah

John Wall 26:03
Another prompt that I had seen run that I just thought I would do is okay let’s marketing over coffee look like and this Gil Elgin motif you know, kind of World War Two propaganda poster kind of thing. And yeah, it’s ridiculous. It’s more like kind of a evil cloning experiment or something. So these will never see the light of day but I thought that was at least funny to to kick out there.

Katie Robbert 26:28
I mean, the top right where one of you is giving like the side eye to the cameras pretty good. I like that. I would, I would love to see you. You know, create, you know, Chris is Chris and John. As what Statler and Waldorf for promo shots remarketing over coffee like that, to me, feels more on brand.

John Wall 26:49
That yeah, I’ll definitely do. And again. So you see the witness? I did. There’s a third coffee mug in here for some oddball reason. I don’t know why that’s Oh, yeah. Yeah, it’s

Katie Robbert 26:59
like knuckling. The handle

John Wall 27:03
is Yeah. Like, the hand is white. And then the background look, this person in the background, not really sure what the heck that is over there. You know, there’s a lot of weird artifacts that show up in these things.

Katie Robbert 27:13
And in the top left, he almost starts to have that like Alien back of the head with like, his hair just kind of keeps going. And it’s almost like that dinosaur that has like, the thing sticking out of the back of his head. I don’t know my dinosaurs well enough to remember that it’s a little odd.

John Wall 27:30
Yeah, like this eyeball over here on the right one, I’m not really sure what the heck’s going on there. So you easily get off track. And it’s still so this one, I went to the Okay, just the absolute objects. And again, latte foam everywhere here. Also, this microphone, I’m not really sure how this actually stands, what the deal with that is, but you know, there it is.

Katie Robbert 27:52
Yeah, that’s a lots of questions. You know, the one on the top right, is maybe the one like I like the way that it has the light coming through the window, like, you know, it seems to do really well, handling lighting, which is sort of giving that depth and dimension, which is really interesting. But yeah, the objects are just odd. And it’s interesting that it’s always choosing a latte with art, when you’re just saying I want a cup of coffee, like coffee

John Wall 28:24
right now, because that dominates the images that are out there. So I grabbed it, we went back to the Impressionist thing, and it was weird, it didn’t go heavy impressionist. And it’s funny, you can see how, you know, when you say in a library, it’s not entirely sure. Okay, is all the stuff in the library? Or is the librarian the phone? Oh, so this over and over as we go, it kind of gives you four different directions. The funniest one with that was, it had a portrait of a couple overlaid over a wolf. And so some of the images were a couple and a wolf, but then two of the variants, one had the guy’s head as a wolf and the other had the woman’s head as a wolf. And that is fantastic. Yeah, isn’t that bizarre? And then, you know, again, it’s just funny to see how you kind of get pushed down these directions, it looks for different options.

Katie Robbert 29:15
Well, and it reminds me of, you know, that old, you know, people use it as an example for like, understanding grammar, is, you know, we, we went to dinner and, you know, ate with grandma or something like it’s something about like, depending on how you put the comma. Its you are either eating your grandmother, or you’re eating with your grandmother. And I feel like a lot of these prompts like you have to understand the rules of grammar. Like in this one is a great example of like, they don’t know where to put the library.

John Wall 29:47
Right. Yeah. And we’ll totally get into that with transcripts. Right, that so this one was good enough. I was like, alright, that could actually work. I could take that and it wasn’t till after I’d rendered it. I was like, Oh, wait. Look, this is funny. It’s an iPhone. And it has the new no button iPhone, but there’s still the silhouette of the button on the screen. Because you know, the older iPhone always had the home button. So it doesn’t, you know, discern between new and old, it just says iPhone, most iPhones have the thing on it. So here it goes.

Katie Robbert 30:18
Well, and of course, and now I’m trying to like now I’m dissecting like the shadows and reflections like, are they all pointing the right way? Or, you know, so this is, this is going to bother me now, I’m gonna have to go back to every single one and see, like, where’s the source of light coming from in this image,

John Wall 30:33
you’ve got full easter eggs, they have a full on plan, you know, for 10 bucks a month, you can get 200 images. So you know that works that again, it’s, you know, cheap when you look at how it goes. And so yeah, throw those out there folks in the chat actually want to place their vote for what they thought was the best of the three, we can let these go toe to toe each other before we go into the transcripts. And I can show you how that stuff went. But yeah, overall, it’s good. The you know, the real thing is, though, you do still need to know how to use some kind of graphics editing tool, because there’s work to be done. It’s not to the point yet where it just automates everything.

Katie Robbert 31:10
Yeah, I would imagine that your prompt would have to include, you know, especially those ones that like include all of the jumbled letters, like with no text, or something like that sort of like that instruction so that it’s not going to give you an image with all this jumbled text that you then have to bring it to camera and try to remove it like it’s it goes back to really understanding how the prompts work. And that’s sort of what this sheet that we created as for TrustInsights.ai slash prompts sheet.

John Wall 31:42
Yeah, it’s all about the prompts. You know, that’ll get we where we want to go. And it is like is it for if you need some kind of weird abstract image that you just can’t get through photography, these tools could at least take you there. Alright, so see, we’ve got votes for three coming in mid journey. Taking the crown there, it looks like I don’t know, though. What do you think, Katie? Which one are you going to pick?

Katie Robbert 32:01
Well, I mean, I like the composition of three, but it’s a cup of tea, not a cup of coffee, and it’s not marketing over tea. That’s the problem. In terms of like, you know, the what you would expect, like the professionalism. I think, number one, I think the Stable Diffusion sort of hits the mark in terms of, we’re a professional podcast, we talk about, you know, marketing, and like we’re serious. And then number two, I like it, because I feel like it would work well, like, as the image for your blog, versus number one could be like the official image of the podcast.

John Wall 32:35
Yeah, yeah, that first one. Definitely. The photography is good. Yeah, it was good to use the Impressionist thing to force it down the right road, but the photography kind of can’t be beat for the corporate field. So

Katie Robbert 32:46
I feel like number three would be good. If you decided to travel to the UK and record some episodes. That’s very, it reminds it’s very British.

John Wall 32:54
Yeah, a very old school library feel to it. That would be right on the art. So yeah, we talked about, again, it’s good to iterate, you know, you’ve got to stay away from the brands, it’s not expensive, you still need graphics design chops. So moving on from there was now on the transcripts, this is actually a different deal. This is like there is guaranteed value here. If you need to generate and transcript these tools can get it done. The first one that I went through was otter AI, we actually have used this for the company for over a year, we have a full on subscription to this. And that was the thing with these is that all of these tools have very different pricing models and use cases. So you know, it’s not as much which one works, but which one can you already jumped on board with? For example, for us, we’ve got a bunch of our conference calls are already being transcribed. So we have records of what goes on. So we can go back. And if we’re not sure about, Oh, what did they say about how their environments set up, we can go back and read it, it’s in there and we can go. But for this it was you know, you basically just go there, you can upload the file, it has a web interface, very simple to use, you upload the thing, it comes back with a transcript preview. This was one thing though, all the tools across the board have different output options. So it depends on where you need it to go and what you’re going to use it for. All of them do dump to text file, so you can at least get there. But there’s other options and other tools which may or may not be better. Otter AI has some interesting stuff in that it does a good job with speaker identification it can figure out who’s speaking which person is there and you can go in just go ahead and change and correct those through the interface. It also now has a summary function where it will actually chop it up into segments so if you want to find out where clips are, it’s not great but it doesn’t like it couldn’t separate out the the mid roll mid roll ads from the content like it couldn’t figure out what was interviewed versus what was ad but it at least you know if you at least knew one of the questions and where you need to be you could jump to there which is great. The other thing with this is this interface is fully interactive too as far as Let me go back to that. So you can check that out the ideas that as you’re I had to cut these down, so they fit in the slides. But at the bottom, you have a textbox. So as you hit the play button, and as the audio plays, it actually highlights the words as it goes. And you can go in and just correct the words that aren’t correct. You know, this is a full working user interface. So you can proofread the whole thing as fast as you can read and then just click playing the segment that’s messed up, it doesn’t make sense. So you can easily hammer through a whole file, and, you know, get it to where you want it to be. And then you dump it to text and go, but they do a good job with this, this is a solid one, the score for these guys. So it took almost 13 minutes to get to the text file. And then it takes longer to label the speakers and do the summary. But 17 minutes, but so we’re not talking about you know, it’s very fast, that’s the, the punch line of that $80 a year they have their pronoun, that’s where on a legacy price, but for less than 10 bucks a month, you can get this tool and generate these. So otter a, I did a solid job with that.

Katie Robbert 36:02
And if you’re like us, and you use it multiple times a day, it’s definitely worth the price. So you know, after this livestream, I will be loading the audio from this particular episode into otter, having it transcribe and I’ll be using it two ways, I’ll be using the text file as the transcript for the blog. And then I will export the SRT file and bring it into my video editor to put in the captions. And what’s nice is to your point, John, because it can identify speakers, it does a really nice job of matching up to the video back to the video with the closed captions.

John Wall 36:38
Yeah, and that’s so we see now with Adobe Premiere, that this is baked into the product. As far as like you said, closed captioning is part of that. The thing with Premiere though, is like you need to understand the Adobe environment, right, which is not a trivial thing. You know, you need to load up all of your media files, you put them in a timeline. And then once you’re at that point, you can just click to generate the transcript, it’s not that complicated, you know, you basically get grabbed by the timeline click to drive through, and it comes out with a transcript. Now they have their own transcript format, actually, that works on other platforms within the environment. But you can also export to txt file, that’s where you get to go with that, right? It takes 13 minutes. So it’s the same neighborhood. It labels the speakers, the weird thing with Premier is that it didn’t do a good job of grouping the quotes, it kept trimming them. So I’ll show you when we see the outputs. But basically, when you look at the otter a five otter AI file, it does a great job at grouping all the text together under single speakers, where this had multiple breaks where it would say, you know, Katy, robear, five or six lines, and that would say, Katie robear, again and have five or six lines, which, you know, it works. But the problem is, if you’re having to go clean that up, you have to trim all that stuff out of there, which is a hassle. And then the premiere is not cheap. We use the Adobe Creative Cloud, I pay 60 bucks a month for access to that. So it’s, you know, that’s a different price point. But another way of looking at is I have to use Photoshop and Dreamweaver. So premiere is free for me, because if I use another tool out of the bin, you know, it’s not costing me more, I’m already paying the full boat. So I get the free rider problem from an economics point.

Katie Robbert 38:26
Yeah, what’s interesting, at least for the two you’ve shown so far, it’s definitely the learning curve of the interface like otter I picked up really quickly, the buttons were very clear, like, there’s a big blue Import button at the top and you click it, and it says, you know, find your file or drag, like so it’s very clear, it gives you like, I am now uploading, okay, I am now transcribing your file is ready. Like it’s very straightforward. I look at Adobe, and I’m not an Adobe user anymore, and I just it’s very overwhelming, just like Discord. It’s like there’s too many things happening where you have to sort of like hunt around the screen to figure out what you even need to do.

John Wall 39:04
Yeah, and it’s also interesting that they’re a lot more agile than they used to be used to be, at least with you learn the tool, you are good. But now they push updates so often, that you know, something you did four months ago may have changed from the next time you go in. And so if you’re spending a good part of every day in there, you don’t care. You know, you adjust after the first day, but God forbid you take two years off from an Adobe Tool and go back and try and figure out how to do something it may not do what you want it to do. I mean, I had some weirdness with the podcast last week, it just starts handling markers differently than it did the week before because they’ve changed some stuff and you can work around it. But yeah, it’s you know, you kind of need to be in that community to make that work. descript was the third one we had heard a lot of good things about it. But this is a whole another completely different take the idea with this is a full on podcasting suite where you use this to record you have your projects in there. And as you drill through, you can if you want, you can do some of the recording from when within some part of the tool. And I have to say I’m like, I’m no expert on this tool, I’ve gone through the free trial for a couple hours. But I, you know, you’re getting the one on one take on this, like I could be missing stuff. But again, you tell it how many speakers you want, it’ll do the transcription. And this was pretty cool. It actually will dump it out in hypertext. You know, if you want HTML, see, that works for me, that would go straight into the show notes, I don’t have to do any work. Like that’s, that’s a step that saves me a few clicks, and some headaches too, because I know what it’s going to look like, I can preview it in there. And then there’s this whole other level of AI on this tool, as far as you know, looking at the transcript and just saying, okay, shorten the word gaps, remove the filler words, you know, it, you basically can edit the audio file by making changes to the transcript. So, even so far as you could sit down, do your intro in your voice, and then go through the transcript and say, Okay, no, don’t do this in the intro, you know that the voiceover voice, and you can change it so that it’s sounds like somebody else is doing your intro for you, even though it’s you. So there was a lot of stuff in this as far as, you know, podcasting tools and nonlinear editing that you can’t get from other any other place. Like if you’re on board with this, you just get this as part of your product. And I have to give them some credit to for a tool that’s, you know, not been around forever. Their help files are a great as far as because like, when I first uploaded, I said it was one speaker, and then I had to go back and have it reprocess it, again, to identify all three speakers, and the instructions to that were good and actually worked. So other platforms, not so much when you get in a hole like that, gotcha. They have a pricing plan, they have a free preview version that gives you, you know, some free stuff. And then if you want to go to the $15 a month program, now you’ve got your whole, you know, creative video and audio podcast thing, they also have transcripts for $2 a minute, if you want to just run it and have somebody else clean it up for you, which may or may not be, you know, if you have a shorter show, and you don’t want to deal with it, this is that’s definitely a good deal. And only five minutes, 18 seconds for them to churn through the file. So it seems like it’s all virtual. So it’s faster, which is, you know, doesn’t affect us too much, but was interesting to know. And then the bonus went to I took the best file the other file and ran through ChatGPT. The short version is it didn’t really didn’t do much of anything. When I used a prompt that said, you know, format proofread and correct, it did a summary of that it took the whole show and just made it into a 10 sentence thing, which, you know, it’s not where I want to be. So then I said, you know, I, and ChatGPT, you it’s actually a running thing, like it remembers the previous one. And you can add, I added do not summarize. And at that point, it just put it back verbatim didn’t actually make any tweaks. So as far as I can see, there’s no value to taking something transcribed somewhere else and running it through ChatGPT is not like it’s gonna make a ton of fixes for you, if you’re going to proofread it, you might as well just proofread it and go.

Katie Robbert 43:08
Well, what’s interesting, though, and you know, obviously, there’s a lot more tools that could have been tested. But in particular, in the three that you looked at, what I didn’t see was a very clean way to summarize it. Because a lot of times, you know, you’re trying to promote the podcast, and, you know, come up with like, you know, 140 characters or less summary of what this episode is about. And so I didn’t see within these tools within the very short, you know, demo that you did, you know, what’s a quick summary or, you know, for them to have a feature that’s like, give me a social appropriate summary of each of these things like this summary from ChatGPT, I could see that being like, at the top of you know, the podcast or like the, like the very top of the show notes before you get into the thing. But otherwise, you know, that, to me is sort of like that’s a big deal. And that’s kind of missing from all of these one stop shops.

John Wall 44:03
Yeah, that’s definitely solid. Because you could use the summary, like you said, for social. And we’ve already seen a bunch of stuff with that was if you’re in a session, and you want to boil it down to points, like that’s the kind of stuff that ChatGPT can really do. Well, if you’ve got that straight. So yeah, that’s it. Yeah, the other two don’t have anything along those lines as far as being able to summarize and give you sound bites, although the which was I was just running through it, you know, it’s like, I can’t even keep the tool straight. I’ve been riding back through so many times. But the script does have the ability to tag stuff as you’re going and say, Hey, this is a highlight. Like I want to do a sound bite on this one as they have that functionality in there on the fly. And you can do it on your transcript afterwards. If you’re reading through, you can highlight sections and say, Hey, this is going to be one of the promotional chunks for the show. So

Katie Robbert 44:59
well Yeah, I mean, and that so you know, some people bulk record. And so like when I do the punch out podcast with Carrie gorgona, we tend to record all the episodes back to back to back over the course of two weeks, versus recording it once a week than releasing it. And so by the time we get to post production, I’ve forgotten what lives within each episode. So if if the tool could highlight for me, like, hey, this could be a sound Boyd soundbite, or this could be part of social like, That, to me would be invaluable, rather than having to go back through all of my notes. And remember what was said?

John Wall 45:35
Yeah, yeah, dig stuff out. Now that’s directly, you know, part of the thing is, because yeah, it’s such a headache. If you can’t even remember now you’ve got to either you listen to it, or get somebody else to listen to it to get the sound bites you just killing yourself there as far as I’m going work. Alright, so to dig into the the results, this was, you know, I had to go through and I got them all down to text and then translated them over to HTML, so I could edit and dig it. And the idea was to have the three of these and be able to go through them all at once, and highlight where mistakes were made, you know, what was wrong and what wasn’t working. As you can see on the left here, the otter AI, one, you know, completely nailed it. As far as grouping the text together, you know, got everybody’s a paragraph in the, the, the speaker switch, and they’re on the mark, whereas, you know, the other two versions were broken the wrong place, like Adobe Premiere, for some reason, really couldn’t tell me in Simon apart that well, like, it would have the first couple of words of Simon, it would still attribute to me. So there’s like, all these extra breaks over here in this file. And then for, yeah, it just, it was just obvious that otter AI dominated as far as getting this done and right. And then there were a bunch of other things, too, that I hadn’t thought about that kind of came out. But to give you the raw results of what actually did well. Otter had 56 errors, descript 73, Premier 84. And then, in addition to the read the errors, I also added a green classification, if there was something that one tool got that the other two got wrong, I gave them a green for that. And so it was interesting to see that premier was actually the best at doing, you know, finding weird odd stuff and getting things like Ted the conference, you know, knowing that that should be all caps, right? Let’s hope that that others didn’t. And then there’s just a whole, you know, list of stuff that I got through as far as, you know, formatting is actually critical taking, you know, and this is so funny, this just goes back to everything else we do with data analysis, right? If the format is formatted, if the file is formatted incorrectly, you have to do a ton of work to get it just to a point where you can figure out if it’s right or wrong, because that’s actually the first order of work. Separating speakers was a challenge. Yeah, and it was even crazy. As far as you know, I would have passages like this edsl paragraph, it was highly technical. And that otter and descript couldn’t tell who was talking and Premier did get it right. But premier got the the actual text wrong, it was you know, the fact that it was at least attributed to the right person was a plus, but it still didn’t get it right. File outputs a big deal, something that works for you. random stuff about grammar and homonyms, right descript uses cuz you see, because that is how people will say, because when they’re going with it, you know, Oh, we did this, because that’s the way it works. But I don’t want cousin my transcripts. Okay, hear that. But I want it to be more formal. And all these tools have files that you can go and say, Okay, here’s some hard to the words that you’re missing all the time. You know, we’re going to train you with these. So hopefully, you get a better and better ability to do that. And otter is intelligent in that it learns speakers over time. So if you do multiple files with a specific person, it will get better at understanding that person and how they do their things. So some interesting machine learning going on for the auto auto platform, which is I think, why they ended up scoring well, overall, above everything. Yeah, punctuation can be subjective. In some cases, it doesn’t make a difference and other care situations. folded arms is much different from folded armed. Yeah, and

Katie Robbert 49:23
very different. Right. That’s like, that’s the whole eat grandma or eat.

John Wall 49:28
Right, right. Eating, eats, shoots and leaves. The end Yeah, in phrases, turns of speech, you know, and phrases like at the end of day, some tools get that, you know, one of the other ones said No, at the end of the dam, which is not a thing. But, you know, again, it’s all these models are different, what they’re trained on. But in summary, you know, all of them are fast they all work and the big thing is, what are you already paying for, you know, if you’re already using otter AI for your company, there’s no reason we’re going to switch right because it totally works. If I didn’t have otter AI, I’d probably use Premiere because I don’t have to pay any more for it. I’m already paying for that. And then yeah, as we talked about with image generation, you know, that’s not really there. This transcript stuff is definitely instant value for you. It’s hard work that it gets done and makes happen. But the image stuff is, you still need to know what you’re doing with images and where to go. But yeah, that was the grand tour. Man, that took a little bit longer than I hoped. But, but we’re there I will open to questions, though. If anybody’s got anything they want to throw in? And I don’t know. What do you think haven’t gone through the roller coaster ride here,

Katie Robbert 50:33
it’s, it’s really fascinating to me, because I’m coming up with new use cases in my brain of things that I would start to look for. So if I was doing my own Bake Off, you know, we started with, Okay, we just want to do transcripts. But now I’m like, Well, what else can it do? You know, so can it do the transcript, but also then clean it up, but also then create, you know, to social posts using the transcript, but then also then, and so I can see where these tools, you know, right now, you know, you have hundreds of spin up startups to do like one individual piece of the puzzle. But the companies that really start to nail doing multiple things like it all in the same context, are really going to dominate the market in terms of meeting the needs of what people have with these tools. And so if, you know, just as an example, if otter can transcribe, but then you can also click a button to say, clean up the grammar, but then also click a button to say summarize it and give me 140 characters. Like, I would pick that all day, every day, because those are the extra steps that I do once I have the transcription. And that would save me a heck of a lot of time and headache, because I’m terrible at writing social content. But if I can have something at least get me started, it’s gonna save a lot of time.

John Wall 51:55
Yeah, getting past the ugly first draft stage is huge. It just can, you know, a lot of people that is the hardest part of writing the thing, and it makes that go away.

Katie Robbert 52:04
Well, I mean, so John, this has been amazing and detailed. And I can’t wait for us to do another version of this. Because this, this has been super informative. And I feel like I learned a lot, especially around the images, because I haven’t used the image systems a lot. And it’s been really helpful to sort of see what the hiccups can be. And so if you’ve been using these tools, and you want to join the conversation, post show and keep it going, you can join our free slack group at trust insights.ai/analytics for marketers, we’re over 3000 Human marketers, not robots, not chat bots, are actually talking about the different tools that they’re using on a daily basis, you can feel free to join us over there. If you want to get more detailed technical information about some of these tools, Chris, and I usually talk about that on the podcast that’s at Trust Insights, or AI slash ti podcast. And if you want to get a summary of a lot of what we’ve talked about today, and then different perspectives, you can sign up for our newsletter at trust insights.ai/newsletter. So John, any final thoughts? I mean, that was amazing. Did you have fun doing that research?

John Wall 53:11
Yeah, it was interesting to play, I think, you know, the image thing is, first, I led with the transcripts, because I knew that that was going to have value. And I ended up moving the images to the front, because part of it was such a, you know, such comedy and some of the stuff that came out there. But it was fun to see how it worked. And it does work. But yeah, it was cool to go through all the things and yeah, I would love to hear in Slack, other platforms that people are using, if you’ve got something that works for you, if you have any tricks as far as images, I would love to hear that. So ping us over there or yeah, we’re Twitter handles are up on here. We would love to hear from you.

Katie Robbert 53:39
Absolutely. All right, John. Until next time, amazing job. Thank you so much. Thank you.

Christopher Penn 53:47
Thanks for watching today. Be sure to subscribe to our show wherever you’re watching it. For more resources. And to learn more, check out the Trust Insights podcast at trust insights.ai/t AI podcast, and a weekly email newsletter at trust insights.ai/newsletter Got questions about what you saw on today’s episode. Join our free analytics for markers slack group at trust insights.ai/analytics for marketers, see you next time.

Transcribed by https://otter.ai

Need help with your marketing AI and analytics?

So What? Marketing Analytics and Insights Live

airs every Thursday at 1 pm EST.

In this episode you’ll learn:

Upcoming Episodes:

One thought on “So What? Generative AI tools bakeoff”

Leave a Reply Cancel reply

Subscribe to our Weekly Newsletter

Pin It on Pinterest