Cutting Through The AI Hype: The Real Business Questions Every CEO Should Be Asking

Back in 2001, my team and I were sitting in a boardroom in Los Angeles, demonstrating what we thought was revolutionary technology. We'd built sophisticated sentiment analysis models that could track public opinion about movies, stocks, and market phenomena in near real-time with 96% accuracy. We were shooting for 97%. It was amazing work – we could predict individual stocks and movie box office sales based on how people were feeling about them.

The executive across the table looked at our demonstration and said, "Yeah, but that one on the screen's wrong."

We tried to explain: "We're 96% correct. We're shooting for 97%. It's better every month."

His response? "Yeah, but that's wrong. I can't do this. I can't work with this."

That moment taught me something crucial about AI that's still true today: the biggest challenge isn't building accurate models – it's understanding what business questions you're actually trying to answer. Twenty-four years later, as I watch CEOs navigate the current AI wave, I see the same disconnect playing out.

Start with revenue, not just cost reduction

When I talk to executives about AI, they usually want to know how it can cut costs. Can we automate customer service? Reduce headcount? These aren't wrong questions, but they're incomplete.

You always want to reduce costs – anything you can automate is good, as long as the automated process is as good as the human-based process. But here's what I've learned working with hundreds of enterprises: while reducing cost is great, the real question is how do you maintain or increase your revenue?

The distinction matters because the tooling and expertise required are fundamentally different. The tools you use to reduce your costs across all your business processes are more commoditized than the tools you use to increase your revenue all the way through your customer pipeline.

You can find plenty of off-the-shelf solutions for demand generation, lead generation, all the way through that pipeline. But when you get to making the business decisions – customer intake, pricing, costing, risk – there's some secret sauce in there that makes your business successful. That's where you need to focus your AI efforts.

As a CEO, your focus should be on topline. Yes, you need to reduce costs to maximize your bottom line, but the techniques and tooling you use for revenue generation need to be different – and more strategic.

Understand the mathematics of business decisions

Before we can really talk about how to deploy AI effectively, I need to take a detour into math.

Here's something that executives without a background in mathematics might not grasp: when you chain decisions together, you accumulate error in ways that can be counterintuitive.

Let me give you a concrete example. If you make a decision that has a 95% likelihood of being correct, then make another decision based on that result that's also 95% correct, followed by a third decision that's 95% correct, your overall accuracy isn't 95%. It's 0.95 × 0.95 × 0.95, which equals about 86%.

When you chain decisions, you accumulate error. This has real business implications.

Take credit processing. You can start with a soft pull on somebody's credit report – it doesn't show up on the credit bureaus as someone applying for credit, and it costs less. But it gives you limited information. Based on that soft pull, you decide whether to do a hard pull, which gives more data but costs more and impacts the customer's credit record. Then you might decide whether human review is needed.

You're following the least cost path to make the best decision possible at each step. But each decision in that chain affects your overall success rate – and your profitability.

Sometimes 95% accurate is excellent. If you're a spell checker, 95% is really bad – that means there could be a wrong word every 20 words. But if you go from a model that gives you 95% accuracy to a model that's 97% accurate, the difference between those two is extremely significant for your business outcomes.

The key insight: you address error accumulation by making each individual decision better and better based on feedback.

This is where AI excels.

Choosing between foundation models and custom solutions

Before diving into AI implementation, you need to understand a fundamental choice that will determine both your costs and your competitive advantage.

When people talk about AI, they usually think about large language models like ChatGPT – those transformer-based models that are extraordinarily expensive to build. But there are lots of other kinds of models.

Places like Hugging Face, Google, Amazon, and Azure have foundation models built on large datasets. One might be for credit card fraud, another for customer churn prediction. These are really great for companies that don't have lots of historical data to create their own modeling.

Then there are bespoke models. If you have data that shows performance over time for a business decision, you likely have what you need to create your own model based on the behaviors you've seen. If your business decisions are different from your competitors – say, determining how risky it is to rent containers for multimodal freight – that might require a custom model.

Here's the decision framework:

• Do you have enough historical data about your specific business outcomes?
• Are your business decisions fundamentally different from industry standards?
• Can you afford the expertise required to build, operate, and maintain custom models?

Most companies don't have the capability in-house to support their own models. Data science is a craft that requires master's level expertise. Software developers may be able to work with a foundation model, but they probably won't understand the underlying issues that data scientists do – things like model quality, model bias, data and prediction drift, and signals that the model needs to be retrained against new data. 

Get a single view of your customers first

Here's a question every CEO should be able to answer immediately: Do we have a single view of our customers across all our touchpoints?

I worked with one company that had 19 different operating units, and every one had a different view of the client. The biggest thing we did was create a single customer ID across the different units.

The consequences of not having this were embarrassing. They would meet with a Fortune 500 company who’d ask, "Okay, what business do we have with you?" The answer was, "I'm not really sure." It only took a couple of these moments for the CEO to say, "Hey, this is a priority. We don't know the answer to this question. We don't know how to sell to these people."

You can't look across the entire customer journey unless you have that understanding across the entire business. Without this holistic view of every customer you're missing opportunities. Knowing everything they've done with you is key. Without that kind of context, you may make a decision that would be completely different if you had the full picture.

Think about a relationship a bank might have with a customer – they may also be that customer’s payroll processor, their credit card company, the customer might use their financial planning, or even have a mortgage with them. This relationship becomes very broad, and you need to see all of it to make intelligent AI-driven decisions.

Ask the right questions before you invest

Rather than getting caught up in AI capabilities and technical specifications, focus on these business fundamentals:

Do we understand our customers completely? If you can't track a customer's entire journey across all your business units, you may not be ready for sophisticated AI implementations. These gaps in understanding are a bottleneck that automation alone can't fix..

What decisions do we make repeatedly that impact revenue? These are your prime candidates for AI enhancement. For example, if a lender needs to make many collections calls a day, who should they call? You want to prioritize based on likelihood to answer and likelihood to pay, given context from previous interactions.

Are we solving business problems or just automating broken processes? Tech doesn't solve problems, people do. You need to optimize the process before automating. Tech problems imply deeper organizational problems.

Do we have the data foundation we need? Do you have lots of high quality information about your customers? Do you have a long tail of information about customer behaviors over time? If you're planning to use third-party information, understand that generally third parties don't want to give you access to enough data to train your own models.

What happens when our current approach fails? Understanding your decision-making process under uncertainty helps identify where AI can add the most value.

Focus on sustainable competitive advantage

I see a lot of people getting excited about AI without thinking through the strategic implications. AI is like any technology – it's neutral. It's how people use it that matters, and that requires intent.

For business leaders, there are two fundamental questions: How do I make my people more efficient? And how do I increase my topline revenue? But here's the thing: you can focus on increasing your revenue and you can focus on reducing your costs, but the approaches are different.

I don't have a big visionary thing for you, but I do know this: before you invest significant resources in AI initiatives, you need to audit your current decision-making processes. You need to ask not what AI can do, but what business problems need solving – and whether you have the data foundation necessary to solve them effectively.

The companies that get this right will discover new ways to incrementally drive revenue that their competitors can't easily replicate. They'll understand that their competitive advantage lies not in having AI, but in applying it strategically to the decisions that matter most to their business.

And that's worth far more than any percentage point of accuracy.

Read the full article on the Ten Mile Square website.