AI Vendor Management: What to Review Before You Buy

Why AI vendors need a different review

Buying conventional software and buying an AI solution do not always involve the same uncertainty.

An AI system may behave differently when its data, model, configuration, prompts, connected tools, features or underlying supplier change.

The organisation may also depend on the vendor to understand capabilities, limitations, tests, risks, monitoring and change.

Due diligence must therefore go beyond the sales sheet. The question is not only does it work? but also can we govern it?

Start with purpose and context of use

Before assessing the vendor, understand the use case. The same tool can have very different risk profiles.

Using a model to draft text, summarise meetings or recommend products is not the same as using it to select candidates, assess people, support financial or health decisions, or operate a critical process.

Define intended purpose, users, affected persons, decisions, data, autonomy and consequences. Risk depends on use, not only on the product, and should connect to AI risk management.

Identify each party’s role

Ask what role each organisation actually performs in the value chain. There may be a system supplier, model provider, integrator, distributor, importer, deployer, subcontractor or cloud supplier.

An organisation may perform more than one role. A commercial contract does not, by itself, determine the regulatory role, and the word vendor should not automatically be treated as provider in the legal sense of the AI Act. Review the relevant AI Act obligations.

Understand what system or model you are buying

Establish whether the purchase is a complete system, a model, an API, an embedded feature, a configured solution or a bespoke development.

Identify third-party components, underlying models, version, relevant architecture, dependencies, limitations, terms of use and anything that can change without customer intervention.

“Uses AI” is not an adequate description. Record it in the AI inventory and, where useful, in a practical AI register.

Data, privacy and confidentiality

Understand what data the supplier receives, where it is processed, how long it is retained, whether it is used for training, which third parties are involved, which transfers occur, how access is controlled and how deletion works.

Also assess what users may enter, particularly personal data, confidential information, intellectual property and trade secrets.

Technical and contractual settings should be consistent with the organisation’s AI use policy.

Risk, security and robustness

The vendor should explain identified risks and applied measures. Review security, availability, robustness, misuse, manipulation, bias, errors, hallucinations and unexpected behaviour.

Also review testing, known limitations, controls, monitoring, adversarial testing where appropriate and recovery mechanisms.

“Our system is secure” is not sufficient evidence. The organisation needs information for its own decision. Nor should every solution be presumed to be a high-risk AI system; classification depends on intended purpose and context through an AI Act classification.

Transparency and documentation

Documentation is critical when an organisation depends on a third party. Relevant material may cover intended purpose, operation, limitations, performance, data, testing, risk, oversight, instructions, versions and change.

Not every technical detail must always be disclosed. Legitimate limits may protect intellectual property, confidential business information and trade secrets.

However, each party needs enough information to discharge its responsibilities.

Human oversight and ability to intervene

Where a system affects significant decisions, determine whether its output can be ignored or reversed, whether the matter can be escalated, whether the system can be stopped, what information the reviewer receives and what record an intervention creates.

Human oversight cannot be designed properly without sufficient understanding of the system and its limitations.

Changes, versions and updates

A major risk appears after purchase: the solution changes. The model, version, interface, underlying supplier, data, functionality, behaviour or contractual terms may all change.

The contract and governance process should define which changes are notified, how much notice is given, which changes require reassessment, what happens when risk changes and what the customer may reject.

The solution originally assessed should not gradually become a different black box.

Incidents, monitoring and support

Due diligence should establish how incidents are reported, within what timeframe, who responds, what information is provided, how investigation works, which logs exist, how remediation is applied and what happens to affected systems.

Define support and escalation channels. An AI incident may require coordination across the vendor, business, technology, security, legal and compliance teams.

Practical checklist before buying

SYSTEM AND VENDOR

• purpose, version, features, limitations and dependencies
• identity, responsibilities, experience, support and relevant third parties.

DATA

• categories, location, retention, training use, access and deletion.

RISK AND SECURITY

• identified risks, tests, controls and residual risk
• access, encryption, vulnerabilities, incidents and continuity.

TRANSPARENCY AND OVERSIGHT

• documentation, instructions, performance and limitations
• human review, override, escalation and stop controls.

CHANGE, EVIDENCE AND CONTRACT

• versions, notice and reassessment
• logs, reports and evaluations
• responsibilities, audit, incidents and exit.

A checklist does not replace analysis. It helps prevent important questions from being missed.

What should be covered in the contract

The contract can become a governance control. Depending on the case, it may address purpose, scope, responsibilities, instructions, information, documentation, assistance, review or audit rights, incidents, change, third parties, data, security, intellectual property, confidentiality, termination, portability and evidence retention.

Certain AI Act value-chain scenarios create specific duties of cooperation, information, technical access or assistance between particular parties.

The contract should enable each organisation to meet its own responsibilities, not make that harder.

For certain high-risk systems and third parties supplying systems, tools, services, components or processes integrated into them, Article 25 of the AI Act contemplates written agreements on the information, capabilities, technical access and assistance needed to support compliance.

Vendor evidence and traceability

Due diligence should leave evidence. Retain the questionnaire, received documentation, risk assessment, decision, conditions, approvals, contract, controls, reviews, incidents and changes.

The outcome may be recorded as approved, approved with conditions, controlled pilot or rejected.

The future question will be: why did we decide to buy this system? The organisation should be able to answer it.

How to integrate vendors into AI governance

The vendor does not disappear after signature.

Inventory

Record the supplier, version and dependencies.

Risk and change

Link system and third-party risks, and reassess material changes.

Monitoring and audit

Receive performance and incident information and review evidence where appropriate.

Renewal and exit

Do not renew automatically without reassessing the context. Define what happens to data, records, evidence, integrations and continuity.

Vendor management is a lifecycle process within the AI governance framework.

Common mistakes

Buying before assessing

The review begins only after the contract has been signed.

Assessing cybersecurity alone

Security matters, but it does not cover every AI risk.

Relying only on certifications

They may provide useful information but do not replace analysis of the use case. ISO/IEC 42001 is not automatically mandatory for vendor selection.

Not knowing which model is behind the service

This can undermine risk and change management.

Ignoring third parties and dependencies

Part of the risk may sit further down the supply chain.

Failing to require change notification

The solution may cease to be the one originally assessed.

Failing to define an exit strategy

Lock-in can itself become a risk.

Failing to retain evidence

The decision becomes difficult to justify later.

Buying AI is also a governance decision

The risk of an AI solution does not end with the vendor. It enters the organisation when the solution becomes part of its processes.

Good due diligence therefore connects purpose, vendor, data, risk, controls, contract, evidence and ongoing monitoring.

The question should not only be do we want this tool? It should also be can we govern it for as long as we use it?

References

• Regulation (EU) 2024/1689 — Artificial Intelligence Act.
• Article 25 — Responsibilities along the AI value chain.
• Article 26 — Obligations of deployers of high-risk AI systems.
• Article 13 — Transparency and provision of information to deployers.
• ISO/IEC 42001:2023 — Artificial intelligence management system.
• ISO/IEC 23894:2023 — Guidance on AI-related risk management.