INBOX INSIGHTS, March 8, 2023: Starting Over, ChatGPT Apps - Trust Insights Marketing Analytics Consulting

INBOX INSIGHTS: Starting Over, ChatGPT Apps (3/8) :: View in browser

Take our Measurement Strategies for Agencies course now for free!

March 2023 One Click Poll

Please click/tap on just one answer – this is our monthly survey to see how we’re doing, so please do take it each month!

How likely are you to recommend Trust Insights as a consulting firm to someone in the next 90 days?

We use this information to measure how effective our marketing is. There’s no form to fill out – tapping your answer is literally all there is to it, and then you get to see Katie’s dog.

Starting Over the Right Way [Title]

You know what really, really sucks? Starting over. Sometimes the best-laid plans really don’t work out and you have to scrap everything and rethink it all.

That’s where I’m at with one of the aspects of the business. Heading into the new year, I had very clear plans on how I wanted to grow Trust Insights. The challenge with the plans is that there were dependencies.

Fast forward two full months into 2023 and the dependencies that I needed to make the growth plans start to happen have fallen apart. It’s maddening. It’s frustrating. It’s demoralizing.

We spend weeks, sometimes months, at the end of each year making plans for the next year. Because we invest all that time, we feel like we have to see the plan through, come hell or high water. The risks and losses become very high.

Why do we do this to ourselves? Make plans, etch them in stone, and then kick and scream when we have to make changes? Because it’s how we’ve always done it.

Maybe the plans that you outlined were fine but they didn’t account for a major disruption, like ChatGPT. Maybe your plans were fine until you lost a key resource. Or, maybe your plans were working until your audience changed their behavior.

Whatever the reason, it’s ok to start over. Let’s do it together, piece by piece.

Do an audit

Before you can fix what’s broken, you have to know where it broke. It may be one piece or all the pieces. In software development, this is a post-mortem. The idea is that you review everything that you’ve done to that point. You discuss what went well and where there is room for improvement. This can be a tough conversation so my #protip to you is to have a mediator. This should be someone that can direct the conversation to ensure that it stays professional and productive. The outcome of this exercise should be a list of things that you need to improve upon.

Make a new plan

If you made it through the post-mortem, you can survive this next part (mostly) unscathed. You have your list of items so you need to start over and come up with a new plan. And then a backup plan to that plan in case the new plan doesn’t work. More plans? Yes. More plans. This is a great excuse to remind you of the simple 5P framework that can help expedite your planning.

The 5Ps are:

Purpose (what is the problem we’re solving?)
People (who needs to be involved?)
Process (how will we solve the problem?)
Platform (what tools do we need to solve the problem?)
Performance (how do we know we solved the problem?)

If you can answer those five simple questions you have built a measurable plan.

Do an audit

Wait a minute, didn’t we do that already? Yes, and we’re doing it again. Auditing your plans regularly will prevent you from having to start all over again. You have a better chance of catching something that is about to go off the rails with your plan when you regularly review. Once you audit, you make a new plan. Then you execute the plan. Then you audit that plan. Then you make a new plan. Then you execute the plan. Then you audit that plan. Then you make a new plan. Then you execute the plan. The key here is consistency and measurability. Plans that you can’t measure will be difficult to audit. Auditing a plan with no measurement is a waste of time.

If you find yourself in a situation where you have to ditch your plans and start over, you’re in good company. It happens to us all. To prevent heartache, set yourself up for success by developing measurable plans and audit them on a regular schedule.

Have you had to start your plans over? Reply to this email or tell me in our free Slack Community, Analytics for Marketers.

– Katie Robbert, CEO

Do you have a colleague or friend who needs this newsletter? Send them this link to help them get their own copy:

https://www.trustinsights.ai/newsletter

In this week’s In-Ear Insights, Katie and Chris talk about social media listening. What is it, and why does it sometimes fall very short of expectations? Learn what social media listening tools can and can’t do out of the box, and why expertise is mandatory to pair with even the best tools on the market. Tune in to find out more!

Watch/listen to this episode of In-Ear Insights here »

Last week on So What? The Marketing Analytics and Insights Livestream, we looked at setting up testing plans for ChatGPT. Catch the episode replay here!

This Thursday at 1 PM Eastern on our weekly livestream, So What?, we’ll be talking with PESO Model founder Gini Dietrich. Are you following our YouTube channel? If not, click/tap here to follow us!

Here’s some of our content from recent days that you might have missed. If you read something and enjoy it, please share it with a friend or colleague!

Take your skills to the next level with our premium courses.

Get skilled up with an assortment of our free, on-demand classes.

In this week’s Data Diaries, let’s talk about ChatGPT’s new API. Announced last week, the API allows technically-sophisticated marketers and companies to perform the same tasks we’ve all had fun experimenting with using the web interface, but at a much greater scale than a single person copying and pasting prompts and results.

What are some of the tasks we might put this new API to work on? One of the tasks that’s been persistently problematic for less advanced tools to handle is sentiment analysis that’s textually aware. Older sentiment analysis methods typically rely on procedures like “Bag of Words”, where words with known sentiment are used to score a passage. “Sucks”, “refund”, “angry” would be examples of such a lexicon. However, this approach is fairly naive and very problematic when it comes to more complex topics.

In general, large language models tend to do much better about scoring sentiment accurately; in production tests, ChatGPT’s model was about 84% accurate with regard to sentiment analysis. That’s a substantial improvement over models using bag of words and similar approaches, which often score in the 70% range for accuracy or lower.

We wanted to test two things: first, can ChatGPT’s API be used efficiently for sentiment analysis at scale, and second, what could we do with the data? We created new code that ingested fifty articles about International Women’s Day (a complex topic poorly suited to simplistic sentiment models) and score them all with ChatGPT.

Let’s take a look at the process. First, we need the content itself, which we extract from the Talkwalker media monitoring system. Note that Talkwalker also includes its own sentiment measure, which either shows -5 (negative), 0 (neutral), or +5:

From here, we load the text of each of the articles into OpenAI’s API with the following prompt programmatically built in as the instructions:

You will act as a linguistics expert. You are familiar with language, semantics, sentiment analysis, tokenization, word order, and other language analysis. You specialize in sentiment analysis and sentiment scoring. Your first task will be to analyze text and return a sentiment score from -10 to +10. If no sentiment score is possible, return 0. Return results in a pipe-delimited format with the score as the first column and the explanation as the second column with a pipe separating the columns.

What comes back is a simple table:

What’s fascinating about the returned data is that not only does ChatGPT return the requested score, but it also returns an explanation of how it derived that score. This is helpful for checking the results, to see if ChatGPT’s model made correct judgements about sentiment.

Now, what could we do with this data? One obvious application would be for content curation; if we wanted to ensure that content shared for the day met a specific standard of sentiment, we could use this data to filter content that met our requirements.

The bigger picture lesson here is one I talked about previously; every prompt that you’ve successfully run in the past in ChatGPT has the potential to be an application, a piece of software in its own right now that could scale far beyond one person typing into a chat box. If you’ve got prompts that you’ve been very successful with, now is the time to think big about what you could do with them at scale.

Here’s a roundup of who’s hiring, based on positions shared in the Analytics for Marketers Slack group and other communities.

Are you a member of our free Slack group, Analytics for Marketers? Join 3000+ like-minded marketers who care about data and measuring their success. Membership is free – join today. Members also receive sneak peeks of upcoming data, credible third-party studies we find and like, and much more. Join today!

We are excited to announce that we’ve got a new mini-course, and this one’s free. It’s called Measurement Strategies for Agencies. You’ll learn the 5 things agencies do most wrong when it comes to developing effective measurement strategies for clients – and how to fix it. It’s just about an hour long, it’s free, and it’s for two groups of people:

People who work at agencies, so you get better at measurement
People who HIRE agencies, so you know what to ask for in your reports

Click here to take this course now for free!

Interested in sponsoring INBOX INSIGHTS? Contact us for sponsorship options to reach over 22,000 analytically-minded marketers and business professionals every week.

Where can you find Trust Insights face-to-face?

PodCamp Philly, Philadelphia, March 2023
Martechopia, London, March 2023. Use MARSPEAKER20 for 20% off the ticket price.
B2B Ignite, Chicago, May 2023

Going to a conference we should know about? Reach out!

Want some private training at your company? Ask us!

First and most obvious – if you want to talk to us about something specific, especially something we can help with, hit up our contact form.

Where do you spend your time online? Chances are, we’re there too, and would enjoy sharing with you. Here’s where we are – see you there?

Our Featured Partners are companies we work with and promote because we love their stuff. If you’ve ever wondered how we do what we do behind the scenes, chances are we use the tools and skills of one of our partners to do it.

Read our disclosures statement for more details, but we’re also compensated by our partners if you buy something through us.

Some events and partners have purchased sponsorships in this newsletter and as a result, Trust Insights receives financial compensation for promoting them. Read our full disclosures statement on our website .

Thanks for subscribing and supporting us. Let us know if you want to see something different or have any feedback for us!

Need help with your marketing AI and analytics?

3 thoughts on “INBOX INSIGHTS, March 8, 2023: Starting Over, ChatGPT Apps”

Leave a Reply Cancel reply

Subscribe to our Weekly Newsletter

Pin It on Pinterest