AI Stability vs. Capability

This data was originally featured in the April 2nd, 2025 newsletter found here: INBOX INSIGHTS, April 2, 2025: Failing Up, AI Stability vs. Capability

In this week’s Data Diaries, let’s talk about AI tool choice and software. One of the most common questions I’m asked at literally every event is what AI software an organization should be using, especially given the pace of change. There are two schools of thought on this, revolving around the two fundamental needs of organizations.

Need 1: Stability

Anyone who’s ever worked in enterprise software knows that stability is mandatory. Software takes time to be certified for things like HIPAA, SOC 2, ISO 27001, and the rest of the alphabet soup of certifications. They’re a delight (I say that with some sarcasm) but they’re essential for working in high stakes situations. As a result, stable software isn’t allowed to change too much.

That’s especially challenging in AI, where the equivalent of Moore’s Law says that AI doubles its capabilities roughly every 7 months. Within the span of a calendar year, AI’s capabilities will be 2x where you started the year.

However, and this is the big however, if you have a problem you can solve with a known model – especially local models like Gemma 3, Mistral, etc. – you can download those, bake them into a system requiring certification, and then freeze it as is. It won’t get smarter. It won’t get better.

But in many situations, it might not need to, if it works predictably and reliably. This is especially useful for any kind of automated task where you don’t want much variance – if you can structure the system around a small, local model like Gemma 3 and have it give reliable outputs, you take the win and you don’t change it until there’s a need to change it.

Need 2: Capability

The second fundamental need for organizations is capability, where you want to use whatever’s the most capable tool at any given time. Use the best, leave the rest – and the best changes on a regular and frequent basis.

Chasing the best is a moving target – for a while, GPT-4 was the king of the hill. Then GPT-4o, then Gemini 1.5, then Claude Sonnet 3.5, then DeepSeek R1, then Claude Sonnet 3.7, and now Gemini 2.5. Each of these models, made by different manufacturers, advanced the field of AI considerably and changed our understanding of what they’re capable of.

When does capability outweigh stability? When you need ever-more advanced capabilities. For example, with most generative AI models, writing a blog post is not something that requires quantum leaps ahead in capability, but writing code does. The more correct, thoughtful, and intelligent code written by AI becomes, the easier it is to build, test, deploy, and maintain.

The same is true for new capabilities. Models that have vision enabled are a huge leap over models who can’t see. If you have applications of AI that would benefit from being able to see the images in a document, through the lens of your phone’s camera, or what you’re doing on screen, stability is worth less than capability and it makes sense to chase capability.

Balancing the Needs

So how do we reconcile these two needs? The answer, unsurprisingly, looks a lot like software development. In good software development, you have three environments:

1. Development, where everything is broken all the time
2. Staging, where broken stuff should mostly be fixed and is being tested for quality and bug removal
3. Production, where everything should work most of the time

As you set up your generative AI infrastructure in your company, it should have similar organization. Your advanced AI pilot group/team is the development environment, chasing capabilities.

As capabilities mature, that group works with your larger AI task force to codify how new capabilities should be used, and settles on a recommended tool set.

And for the broader users in your company who don’t need the cutting edge, you stabilize on known good tools and methods.

If you organize your AI efforts similar to how you manage other software – with an acknowledgement that even the production setup may change faster than traditional software – you’ll be able to balance the needs of capability and stability more easily.

Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.