INBOX INSIGHTS, August 3, 2022: A/B Testing Failures, Instagram Engagement, Aptitude in Hiring - Trust Insights Marketing Analytics Consulting

INBOX INSIGHTS: A/B Testing Failures, Instagram Engagement, Aptitude in Hiring (8/3) :: View in browser

Take our new Google Analytics 4 for Marketers Course »

What Happens When Your A/B Test Doesn’t Give You the Answer?

This may or may not come as surprise, but I am never satisfied with things if they are static. I feel like most of you out there can agree with that sentiment.

Recently, I wanted to shake up our website traffic and see if I could generate more conversions. I designed and ran an A/B test. I let it run for 30 days. I would check every few days to see who was winning, A or B. You know what happened at the end of 30 days?

Nothing.

The test ran, it ran correctly. But the results were inconclusive. Each option, both A and B were equally engaging (or not engaging) to website visitors. Which means I don’t have the answer I was looking for. Where do you go from here? What happens when your A/B test doesn’t give you the answer?

The good news is you always have options.

Re-run the test
Run a new test
Run a test that isn’t A/B testing
Leave things as they are

Let’s unpack these, shall we?

Re-run the test

You can re-run the test you just ran to see if you get different results. Why should you consider this? You may have a seasonality to your site (or ads, or email) that wasn’t factored in. For example, I let my A/B test run for the month of July. Historically, that’s one of our slower months. Had I realized that earlier in the year I probably would have slated a different time frame. Take a look at your data before deciding the timing of an A/B test. That might make a difference.

Run a new test

This one seems pretty obvious. If the first test was inconclusive, run a new test. You can design something completely different than the first one, or use the same control and test a new experiment. For example, I was testing part of our home page to see if changing the content would result in more conversions. When I design a new test, I’ll use that same section of the homepage but with different content, with the same goal of more conversions.

Run a test that isn’t A/B testing

This is more along the lines of market research, so not really a test at all. Instead of a programmatic A/B test using software like Google Optimize, ask your customers and community what they want. These are people you already know and who know you, versus anonymous people visiting your site. You can show them the existing asset and then give them different options. Ok, it’s an A/B test. I almost had you though.

Leave things as they are

You know what? You don’t have to change anything. If you run an A/B test and the results are inconclusive, you don’t necessarily need to start making other changes. Maybe it was the wrong test and the wrong time. Give it a minute. Breathe a little. Go back to your data and see if there is something different you should be looking at. Maybe it’s not the asset, but the audience or the offer that’s wrong.

At the end of the day, more testing is always a good idea. Test early, test often. Behaviors change, platforms change, and solutions change.

Are you A/B testing your marketing?

Come tell me about it in our free Slack group, Analytics for Marketers.

– Katie Robbert, CEO

Do you have a colleague or friend who needs this newsletter? Send them this link to help them get their own copy:

https://www.trustinsights.ai/newsletter

In this week’s episode, Katie and Chris tackle the question of how to determine if someone has a good attitude and aptitude for marketing, business, or any professional even if they don’t have professional experience. The labor market is constrained for “hit the ground running” candidates, but the actual labor pool is much, much larger – and candidates like veterans, returning mothers, and other non-traditional folks could be even better than the current talent if given the right support. Tune in to find out how to determine whether someone’s a hidden gem.

Watch/listen to this episode of In-Ear Insights here »

Last week on So What? The Marketing Analytics and Insights Live show, we dug into accessibility and website performance. Catch the replay here »

This Thursday at 1 PM Eastern, we’ll be doing a MarTech AMA. Are you following our YouTube channel? If not, click/tap here to follow us!

Need a reminder? Click here for a calendar appointment:

Here’s some of our content from recent days that you might have missed. If you read something and enjoy it, please share it with a friend or colleague!

Take your skills to the next level with our premium courses.

Get skilled up with an assortment of our free, on-demand classes.

Data Diaries - Interesting Data We Found

In this week’s Data Diaries, let’s do a dive into some social media data and revisit our old friend Instagram. Instagram’s gone through a lot of changes this year, as the service has forced Reels upon pretty much everyone. In the data API, Reels are classified as videos; has that had any impact?

Let’s first take a look at brands. How have brands been faring lately?

By now, this is a familiar, if unwelcome story: brands’ performance on Instagram continues to slip away more every month. By month’s end, it’s not unreasonable to guess that brand performance on Instagram will be 0.1%; for every 1,000 followers a brand has, you can expect 1 to engage with your content.

Next, let’s examine content types:

What we see is… well, not very much different. Carousels – albums – still tend to do the best for brands, while videos (even the new Reels) don’t perform especially well.

What about influencers, individual accounts? Let’s take a look.

Influencer engagement follows a similar path, though it’s worth noting that performance has declined more in the past 3 months. Influencer engagement now hovers around 0.4%; this is a sharp decline from even two years ago when it was around 2%. What about content types?

Influencer engagement by content type on Instagram

We saw a brief bump in video performance in May and June, which has receded. Of note, individual photos have declined much more substantially than album, though for a brief period in June they slightly outperformed albums.

So what? What do we take away from all this?

Instagram has two major problems for brands – well, for anyone, really. First, from a strategy perspective, Meta (the parent company of Instagram) is trying really, really hard to be TikTok, but users don’t want a TikTok clone – and those who do just spend their time on TikTok instead. That means other types of content – content that brands have traditionally been more comfortable with posting, like static images – get deprioritized in favor of Reels.

Second, Instagram is a very, very saturated market. Tons of content, but the user base isn’t spending as much time there – a competitive threat Meta has mentioned in its quarterly earnings calls. Too much supply and not enough demand is a losing combination for brands who want attention on their content.

What’s a marketer to do? Well, here’s what we’re doing. First, we’ve stopped posting ordinary videos. We now post as Reels – and their performance is anywhere from 10-100x a regular video. Second, the short form video format is portable. You can take that short form content and put it up as a Reel on Instagram, as a standard piece of content on TikTok, and as a short on YouTube.

But that means you need to be producing video, period. If you’re not, then as Katie says, today is the best day to get started. Whether we want to or not, whether we’re comfortable with it or not, it’s what a significant portion of the audience we want to reach is using to consume content. As the audience goes, so must we if we want to retain their attention.

This is a roundup of the best content you and others have written and shared in the last week.

SEO, Google, and Paid Media

Content Marketing

Data Science and AI

Are you a member of our free Slack group, Analytics for Marketers? Join 2400+ like-minded marketers who care about data and measuring their success. Membership is free – join today. Members also receive sneak peeks of upcoming data, credible third-party studies we find and like, and much more. Join today!

We heard you loud and clear. On Slack, in surveys, at events, you’ve said you want one thing more than anything else: Google Analytics 4 training.

We heard you, and we’ve got you covered. The new Trust Insights Google Analytics 4 For Marketers Course is the comprehensive training solution that will get you up to speed thoroughly in Google Analytics 4.

What makes this different than other training courses?

You’ll learn how Google Tag Manager and Google Data Studio form the essential companion pieces to Google Analytics 4, and how to use them all together
You’ll learn how marketers specifically should use Google Analytics 4, including the new Explore Hub with real world applications and use cases
You’ll learn how to determine if a migration was done correctly, and especially what things are likely to go wrong
You’ll even learn how to hire (or be hired) for Google Analytics 4 talent specifically, not just general Google Analytics
And finally, you’ll learn how to rearrange Google Analytics 4’s menus to be a lot more sensible because that bothers everyone

With more than 5 hours of content across 17 lessons, plus templates, spreadsheets, transcripts, and certificates of completion, you’ll master Google Analytics 4 in ways no other course can teach you.

Click/tap here to enroll today »

Interested in sponsoring INBOX INSIGHTS? Contact us for sponsorship options to reach over 22,000 analytically-minded marketers and business professionals every week.

Where can you find Trust Insights face-to-face?

MAICON, August 2022, Cleveland, OH, USA – use code PENN150 for $150 off any conference ticket
Content Marketing World, September 2022, Cleveland, OH, USA
MarketingProfs Friday Forum, September 2022, Online
MarketingProfs B2B Forum, October 2022, Boston, MA, USA
Heapcon, November 2022, Belgrade, Serbia

Going to a conference we should know about? Reach out!

Want some private training at your company? Ask us!

First and most obvious – if you want to talk to us about something specific, especially something we can help with, hit up our contact form.

Where do you spend your time online? Chances are, we’re there too, and would enjoy sharing with you. Here’s where we are – see you there?

Our Featured Partners are companies we work with and promote because we love their stuff. If you’ve ever wondered how we do what we do behind the scenes, chances are we use the tools and skills of one of our partners to do it.

Read our disclosures statement for more details, but we’re also compensated by our partners if you buy something through us.

Some events and partners have purchased sponsorships in this newsletter and as a result, Trust Insights receives financial compensation for promoting them. Read our full disclosures statement on our website .

Thanks for subscribing and supporting us. Let us know if you want to see something different or have any feedback for us!

Need help with your marketing AI and analytics?

One thought on “INBOX INSIGHTS, August 3, 2022: A/B Testing Failures, Instagram Engagement, Aptitude in Hiring”

Leave a Reply Cancel reply

Subscribe to our Weekly Newsletter

Pin It on Pinterest