9 Questions to Ask an AI Vendor Before You Sign

Why the demo is the wrong thing to judge#

A demo shows you the vendor on its best day, with its own data, run by the person who built it. None of that will be true once you sign.

What decides whether the deal works is in the paperwork: who can use your data, what the bill looks like in month six, and what happens when something breaks. Those questions are easy to ask and awkward to answer, which is exactly why they get skipped.

Here are nine. You do not need a technical background to ask any of them. You need to insist on a written answer.

If you want these on one page to take into the call, there is a printable version.

Question 1: Do you train your models on our data?#

Ask it in exactly those words. Then ask for the clause in the contract that says so.

This matters more than it sounds. In one review of enterprise AI contracts, 43% did not specify whether customer data could be used for general model training. Silence in a contract usually favours the vendor.

What a good answer looks like. A written ban on training with your data, in the data processing agreement. That is the contract that sets out what the vendor may and may not do with what you send it.

Also ask about retention. Retention is how long they keep your data after using it. Many large providers offer a zero-retention setting, but it is rarely on by default. In one analysis, retention windows went unmanaged in 60 to 70 percent of agreements, even where zero retention was available. If it exists, ask for it in writing.

Question 2: Where does our data go, and who else touches it?#

Your data rarely stays with the company you signed with. It passes through subprocessors: the other companies your vendor pays for hosting, storage and, often, the AI model itself.

Ask for three things:

1. Which countries your data is stored and processed in.
2. The full list of subprocessors, by name.
3. How you will be told when that list changes.

Why it matters. If you have customers in Europe, or you work in health or finance, where data travels is a legal question, not a technical one. A vendor that cannot name its subprocessors cannot promise where your data ends up.

Question 3: Can you show us why the model gave a specific answer?#

Do not accept a general description. Pick a real output from the demo and ask them to walk you through how it was produced.

If they cannot, then "explainable" is a sales word, not a feature.

When this becomes a legal requirement. If your product makes decisions about people, such as hiring, lending, insurance or healthcare, and you serve users in Europe, the EU AI Act applies. General application of the Act began on 2 August 2026. The heaviest fines, for prohibited uses, reach €35 million or 7% of global turnover, whichever is higher.

The trend is going the wrong way. Stanford's Foundation Model Transparency Index scores how openly the big AI developers disclose how their models work. In its 2025 edition, the average score fell to 41, down 17 points on the year before. Transparency is something you have to ask for.

Question 4: How do you test for accuracy and bias, and how often?#

Ask for the testing document itself, not a summary of it. Three things should be in it:

• What they test against. The set of examples they use to check the model's answers.
• How often they retest. Models change, and a result from last year says little about today.
• What happens when they find a problem. Who is told, and how fast it is fixed.

Why this deserves scepticism. The people who work with these tools every day are not convinced either. In Stack Overflow's 2025 developer survey, only 29% of developers said they trust AI tools, even as 84% were using or planning to use them.

Question 5: What will year one actually cost, line by line?#

The licence fee is the number on the proposal. It is not the cost.

One analysis puts licence fees at 20 to 40% of the true cost of an AI system. The rest is work around it. Ask the vendor to itemise every one of these before you negotiate the headline price:

• Usage charges. Many AI tools bill per use. A quiet pilot and a busy month can produce very different invoices.
• Integration. Connecting the tool to your existing systems is usually your cost, not theirs.
• Training your team. Someone has to learn it, and that time is spent.
• Keeping it running. After launch, maintenance and support add up. Some estimates put that work near 65% of the system's lifetime cost.

What a good answer looks like. A written estimate for year one that includes usage at the volume you actually expect, not the volume in the pilot.

Question 6: What does it cost to leave?#

Ask this before you are tempted to stay. Three parts:

1. Can we export all our data, in a format we can use elsewhere?
2. How much notice do you need, and is there a fee to leave early?
3. If we build on your tool, how much of that work do we lose when we go?

Why it matters. Switching is harder than most buyers expect. In an IBM Institute for Business Value study, more than 7 in 10 leaders said switching from their primary AI provider would be challenging. The best time to negotiate your exit is before you depend on them.

Question 7: Can we see your security paperwork?#

Ask for documents, not reassurance. The ones that matter:

• A SOC 2 Type II report. An independent auditor checks that the vendor's security controls actually worked over several months, not just on the day of the audit.
• A signed data processing agreement, the contract from Question 1.
• Their approach to AI governance. For larger buyers this increasingly means ISO 42001, a standard for managing AI systems responsibly, which is moving from a nice-to-have to a vendor requirement.

Why this is not paperwork for its own sake. Your vendors are part of your attack surface. According to Verizon's 2025 breach report, 30% of breaches now involve a third party, double the 15% a year earlier.

If a vendor cannot produce these documents, that is the answer.

Question 8: What does your uptime promise actually pay out?#

An SLA, a service level agreement, is the vendor's written promise about how often the service will be available. The number sounds precise. What matters is what happens when they miss it.

The major AI providers publish SLAs between 99.5% and 99.95% uptime. Those decimals are not small:

Promised uptime	Downtime allowed per year
99.5%	about 44 hours
99.9%	under 9 hours
99.95%	under 4.5 hours

Ask what you get when they fall short. Usually it is service credits, a discount on a future bill, not compensation for your lost business. Credits are also often only issued for outages longer than an hour. If your product stops when their service stops, plan for the outage yourself rather than relying on the credit.

Question 9: What happens to us if you go out of business?#

This is uncomfortable to ask a startup. Ask anyway.

It is not hypothetical. Builder.ai, a heavily funded AI app-building company, entered insolvency proceedings in May 2025, leaving its customers to work out what they could recover.

What to ask for:

• Some evidence of stability. How long their funding lasts, or whether they are profitable. They may not share exact numbers, but a refusal to discuss it at all is informative.
• A written right to your data if the company closes or is sold.
• An export you have actually tested, not one described in a sales call.

How to compare the answers#

Without a structure, the vendor with the best salesperson wins.

Reading the diagram: Compare written evidence rather than presentation quality. Group the nine questions into data and security, quality and economics, and continuity and exit. Record what was supplied, what remains unclear, and who owns the follow-up. Read the source: NIST: AI Risk Management Framework.

Write the nine questions down the side of a page and each vendor across the top. Score each answer from 1 to 5, where 1 is "no answer" and 5 is "a document proving it." Decide in advance which questions matter most to you. For most founders that is data first, then cost, then how to leave.

Then do it again later. Problems with a vendor usually surface six to twelve months in, after the pilot is declared a success and the real workload arrives. A vendor that scored well before signing and badly at six months is telling you something no contract clause will fix.

What to do before your next vendor call#

1. Send the nine questions ahead of the meeting, in writing. Good vendors arrive with answers.
2. Ask for documents, not descriptions, for Questions 1, 2, 7 and 8.
3. Get a year-one cost estimate at realistic usage before you discuss price.
4. Test the data export during the trial, not after you have decided to leave.
5. Treat a refusal as an answer. You are not being difficult. You are doing the part of the purchase that protects you later.