On this page
AI proposals tend to follow a familiar arc. The deck is polished, the timeline is ambitious, and the language about transformation and ROI is confident. Then the project starts, the scope expands, and six months later you're explaining to your board why the thing still isn't in production. RAND Corporation found that more than 80% of AI projects fail to deliver their intended business value, roughly twice the failure rate of comparable IT projects. MIT's Project NANDA put an even finer point on it: 95% of organizations see no measurable return to the income statement from their generative AI pilots. The proposals aren't always dishonest. More often, they're built on assumptions nobody checked before the contract was signed. You don't need a data science background to catch the warning signs.
Why AI Proposals Are Hard to Evaluate
Evaluating an AI proposal is different from evaluating a standard software purchase. Traditional software has well-understood implementation patterns. You know what an ERP rollout looks like. You know what a CRM migration entails. AI is different, and the proposals that land on your desk reflect that difference, for better or worse. Enterprise software delivers value through feature availability: the vendor either has the capability or it does not. AI delivers value through deployment quality. The vendor either can build, integrate, govern, and scale a system in your environment, with your data, alongside your existing workflows, or it cannot. A demo reveals the first dimension; it reveals almost nothing about the second.
This is the gap that burns buyers. The demo works. The proposal reads well. And the real work, data prep, integration, change management, monitoring, either wasn't scoped or was quietly assumed away. The six questions below are designed to surface that gap before you sign.
Question 1: What specific business problem does this solve?
A well-constructed AI proposal starts with a business problem and works backward to the technology that solves it. A weak proposal does the opposite, starting with a technology (large language models, computer vision, AI agents) and then searching for a justification. If you can't identify the problem in a single measurable sentence after reading the proposal, that's not a knowledge gap on your end. The proposal isn't ready.
Ask the vendor to restate the problem in plain terms: what process breaks today, what it costs, and how the solution changes the number. If they can't do it, the engagement hasn't been diagnosed, it's been pitched. This is the single most reliable early signal.
We've seen it play out the other direction too. A medical system came to us needing to connect to data from a legacy system. An overseas firm quoted six weeks on a solution that assumed an API existed. It didn't. Understanding the actual problem first, in that case, that a screen-scraping approach was the only viable path, meant delivering a working solution in two weeks instead of six. The problem defined the approach, not the other way around.
Question 2: What does the timeline assume about your data?
Timeline is where most AI proposals quietly fall apart. When a vendor quotes a fixed-scope AI project without completing a data readiness assessment, that is a red flag. Data preparation routinely consumes the majority of a project's calendar time, and vendors who ignore it are either inexperienced or optimistic in ways that will cost you. The failure is almost never the model. It is data readiness, workflow integration, and the absence of a defined outcome before build starts.
Ask specifically: has anyone looked at your actual data yet? What format is it in? Where does it live? How clean is it? If the vendor can't answer those questions before scoping, their timeline is a guess dressed up as a plan. See also our post on what clean data actually requires before an AI project can succeed, the data foundation work often takes longer than the model work.
Question 3: How is success defined, and who measures it?
Vague success metrics are one of the most common structural weaknesses in AI proposals. Many consulting firms structure contracts to deliver documentation and recommendations, not outcomes. Before signing, make sure the contract specifies measurable acceptance criteria tied to the firm's payment milestones.
Push for specifics. What metric moves? By how much? By when? Who owns the measurement? If the answer involves phrases like "improved efficiency" or "better decision-making" without a number attached, you don't have a success definition, you have a marketing sentence. 77% of AI project failures are organizational in nature, poor adoption, undefined ownership, or misaligned success criteria. Getting a number on paper before work starts is how you protect against a project that technically "delivered" while changing nothing.
Question 4: Who actually builds this, and who supports it afterward?
Ask for names and LinkedIn profiles of the engineers who will actually work on your project. Check their actual experience. If the people you meet in the sales process are not the people who will do the work, that gap will cost you time and money. Subcontracting is common in this space, and it rarely gets disclosed upfront.
The post-launch question matters just as much. Models degrade over time as real-world data patterns change. If a firm does not have a documented answer to how they monitor, retrain, and redeploy models post-launch, your AI project will require a second engagement in 12 months. A proposal that ends at go-live is half a proposal. Ask what the support model looks like at 3 months, 6 months, and a year out, and get it in writing.
This is part of why long-term partnerships tend to outperform one-off project shops. A partner who has been inside your systems since 2018, for example, doesn't need to spend the first month just learning your environment before they can scope accurately. They already know where the bodies are buried.
Question 5: What are you not being told about cost?
The license fee is the smallest part of what an AI vendor will cost you. Connecting an AI system to your existing data sources, authentication systems, workflows, and monitoring infrastructure is typically 2 to 5x the license cost in the first year. Someone also needs to monitor the system, triage errors, handle escalations, retrain users, and manage the vendor relationship. That is ongoing headcount cost that never appears in a vendor proposal.
Ask for a fully-loaded cost estimate: implementation, integration, internal staff time, training, and ongoing maintenance. If the vendor resists putting that on paper, you're looking at a number that was designed to win the deal, not fund the project. Cloud costs follow the same pattern. If your AI workload runs on infrastructure that wasn't purpose-built for it, you can end up with significant overspend that nobody flagged at proposal time, the same dynamic we see in cloud audits where clients discover they've been running expensive resources at 2% utilization. Check out our post on cloud cost waste if you want to see what that looks like in practice.
Question 6: What happens if this doesn't work?
Most AI proposals have no honest answer to this question. Ask it anyway. What are the exit criteria? If the proof of concept doesn't hit the agreed metrics, what happens next, does the engagement pause, does money come back, does the scope change? The most reliable red flags are guaranteed outcomes promised before any diagnostic work, and refusal to define success criteria before a proof of value begins. Both are signals of accountability avoidance.
Unless you see a product working live with your data, it's hard to be sure it actually works and can handle your specific workflows. For a demo of a sales order entry product, for example, come prepared with real data and ask the vendor to process it live. If the vendor is reluctant or says they'll send results later, that's a clear red flag. A vendor who can't talk plainly about failure modes is telling you something about how the relationship will go when something actually goes wrong. The conversation to have before signing is much cheaper than the one you'll have six months in.
A Quick Reference: Six Questions to Bring to Any AI Proposal Review
- What specific business problem does this solve, stated in a single measurable sentence?
- What does this timeline assume about our data, and has anyone actually looked at it yet?
- How is success defined, what number moves, and who is accountable for measuring it?
- Who are the engineers doing the actual work, and what does post-launch support look like?
- What is the fully-loaded cost, including integration, internal staff time, training, and ongoing maintenance?
- What are the exit criteria if the proof of concept doesn't hit the agreed targets?
You don't need to understand how transformers work to evaluate an AI proposal. You need to know whether the vendor has done the diagnostic work, and whether they're willing to be accountable for results, not just delivery. If the answers to these six questions are vague, that's your answer. For a deeper look at how to think about build-versus-buy-versus-partner tradeoffs once you do find a vendor worth trusting, this post walks through the decision framework in detail.
If you want a second set of eyes on a proposal you're currently evaluating, or want to understand what a scoped, accountable AI engagement actually looks like, our AI services practice is where we do that work. We start with the diagnostic, not the pitch.
Not sure if the proposal on your desk holds up? We'll give you a straight read. Talk to us