In short: An advisory board can spot whether an AI investment has substance by six markers: a concrete business problem, a baseline measured beforehand, known costs per completed case, a solid data foundation, a plan to move from pilot to production and a named owner. If several are missing, it’s more likely busywork. Funds should therefore be released in stages, tied to measurable milestones.
Why is evaluating AI investments so hard?
AI initiatives almost always look convincing in the demo. The problems only surface in production, for example with poor data or costs that grow with usage. Gartner notes that additional AI features often increase costs without improving results, and that better results require systems that hold the company’s knowledge and context (Gartner, July 1, 2026).
Then there’s a knowledge gap. In Deloitte’s “Governance of AI” study, two-thirds of respondents say their board has limited or no AI knowledge (Deloitte, 2nd edition). A board that doesn’t trust itself to evaluate the technology tends toward one of two reactions: it waves things through because AI sounds strategically important, or it puts on the brakes because it can’t assess the risk. Both can be avoided with the right criteria.
What criteria should an advisory board apply to AI investments?
A concrete business problem.The investment starts with a bottleneck, a cost item or a growth opportunity, not with a tool. “We’re rolling out copilots” is not a problem. “Our quotes take eleven days and we’re losing deals” is.
A measured baseline.If you don’t measure beforehand, you can’t prove success afterward. The metric has to be captured before the project, using the same methods that will be used later.
Known costs per case.AI creates variable costs that grow with usage and with the difficulty of the cases. The question is what a completed case costs in production, in the simple case as well as the hard one.
A solid data foundation.The data the system draws on exists, is maintained and can legally be used. If that’s not the case, the data work belongs in the investment proposal.
A path from pilot to production.It’s spelled out what happens after a successful pilot, who takes over operations and how the system scales. A pilot without that plan is an experiment, not an investment.
A named owner.An executive from the business, not from IT, owns the result and reports to the board.
Which warning signs point to busywork?
- The number of pilots is reported as success. Twenty pilots without a single production deployment aren’t progress. They’re a cost center.
- Lots of tools, no architecture. Every department has its own AI tool, and nobody knows which data flows where or what it all costs.
- AI is handed out for free. Features are rolled out broadly without anyone knowing the ongoing cost per use.
- The benefit is described in hours nobody actually saves. “Two hours per employee per week” is only a return if it’s clear what happens with that time.
- Responsibility sits with IT alone. Then AI is run as a technology project, and nobody owns the impact on the business.
How should an advisory board release funds for AI?
Staged approval has proven its worth. It guards against two mistakes at once: betting everything on one card and starving promising initiatives.
| Stage | Funds | Release of the next stage when … |
|---|---|---|
| 1. Trial | small share of the budget, a few weeks | the system measurably improves the baseline in real operations with real data |
| 2. Pilot in production | medium share, one or two business units or customers | cost per case, employee acceptance rate and error rate are within the target range |
| 3. Scaling | bulk of the funds | operations, ownership, training and cost management are in place |
The stages change the quality of the conversation in the boardroom. Instead of debating strategy in general terms, the board checks the same metrics at every stage. That makes progress visible and makes stopping a project legitimate.
A pilot that was stopped with clear findings is a better outcome than one that keeps running without them.
How do strategic AI investments differ from operational ones?
Not every AI investment has to pay off in the first year. Investments that protect or reposition the business model, such as in-house AI agents at a software company, often weigh on earnings for several years. They need a different kind of evaluation than efficiency projects. The key is not to mix the two. For strategic initiatives, a board should ask for a separate view in which the metrics and margin of the new business are reported on their own. Measure them against the yardsticks of the core business and you’ll shut them down too early.
Frequently asked questions
How do you measure the success of an AI investment?
With a business metric captured before the project, such as cycle time, cost per transaction, error rate or revenue per customer, plus the ongoing cost per completed case. Hours saved alone are not proof.
Why do so many AI pilots fail?
Common reasons are inadequate data, missing baselines, unclear ownership and no plan for production. Many pilots work with hand-picked cases and fail when they meet the breadth and difficulty of day-to-day business.
Should an advisory board approve AI investments individually?
That depends on the articles of association and the list of transactions requiring approval. Either way, a board should ask for staged approval with clear criteria for major initiatives and review progress regularly against the same metrics.
Further reading: Five questions an advisory board should ask management about AI · EU AI Act: what advisory and supervisory boards need to know · Governance starts with trust
Sources: Gartner press release, July 1, 2026 · Deloitte (2025): Governance of AI, 2nd edition