What it means to monitor an AI system
Monitoring checks whether a system still serves its purpose in real conditions and triggers action when its performance or context changes. Data, users, affected people and product versions can change after approval. Start with an AI inventory, intended purpose and legal classification.
Monitoring versus post-market monitoring
AI monitoring is a broader organisational practice for quality, safety and AI governance, suitable for systems according to their risk. Article 72 post-market monitoring is a specific legal duty for providers of high-risk AI systems. Using and monitoring a third-party tool does not by itself make an organisation its provider.
Post-market monitoring under the AI Act
Under Article 72 of Regulation (EU) 2024/1689, consolidated text, the provider of a high-risk AI system must establish and document a system proportionate to the technology and its risks. It must actively and systematically collect, document and analyse relevant performance data from deployers and other sources throughout the system's lifetime. The purpose is to assess continued compliance with Chapter III, Section 2 requirements.
The system must follow a post-market monitoring plan forming part of the technical documentation (Annex IV). Following Regulation (EU) 2026/1744, Article 72(3) calls for Commission guidance including a voluntary template by 2 September 2027. As of September 2026, do not treat that template as an already published mandatory format. High-risk rules have phased application dates: check the system category and current timetable before asserting a duty is in force.
What data should be monitored
| Area | Useful signal | Possible response |
|---|---|---|
| Performance | Accuracy, recall, F1, error rate, availability or latency | Compare with validated baseline |
| Output quality | False positives, false negatives, incomplete or inconsistent results | Review samples and potential harm |
| Data | Distribution, representativeness, missing values or anomalous inputs | Check origin and quality |
| Risk | New groups affected or weakening controls | Reassess risks |
| People | Corrections, overrides, escalation and complaints | Review instructions and oversight |
| Compliance | Changes to use, documentation, duties or incidents | Update controls and evidence |
Metrics depend on purpose and potential harm; there is no universal dashboard.
Drift: when system behaviour changes
Data drift means inputs change; concept drift means the relationship between inputs and outcomes changes; performance drift means measured results deteriorate. A new applicant population may shift input distribution without proving a failure. Compare samples, labels, context and outcomes before drawing conclusions. Drift is a warning signal, not automatically a serious incident or legal breach.
KPIs, KRIs, thresholds and alerts
KPIs track operation, such as accuracy, latency, availability and automation rates. KRIs flag risk, such as complaints, human overrides, false positives, out-of-range results and distribution shifts. Choose them by purpose, users and affected people; see our KPI and KRI guide.
A practical process sets a baseline, threshold, owner and response in advance: KRI exceeds threshold → review → risk assessment → recorded decision. This is an example of good governance, not a universal legal threshold or frequency.
Human oversight and traceability
Tracking recommendations reviewed, corrected, overridden or escalated tests whether human oversight works and whether staff over-rely on outputs. Interpret changes in context: more overrides could also mean better attention.
Logs and traceability support logs → data → monitoring → detection → evidence. Distinguish technically available logs, specific Article 12 and 26 requirements, and voluntary organisational records. Set access and retention according to privacy duties and the situation.
Monitoring duties of deployers
Article 26(5) requires deployers of high-risk AI systems to monitor operation on the basis of instructions for use and, where relevant, inform providers under Article 72. Where they have reason to consider that compliant use may present an Article 79(1) risk, they must inform the provider or distributor and the market surveillance authority without undue delay and suspend use. Article 72 separately places its post-market monitoring system and plan on the provider.
Risks, changes and incidents
Article 9 makes risk management for high-risk systems continuous and iterative throughout their life cycle, informed by post-market monitoring data. In practice: initial risk → deployment → signals → reassessment → controls.
Monitor model version, provider, data, configuration, prompts, integrations, purpose and affected population. Changes may warrant validation, change management, reclassification or updated documentation. An update is not automatically a legally substantial modification. Monitoring detects signals; AI incident management investigates and responds. Article 73 addresses provider reporting of serious incidents involving high-risk systems where applicable; an ordinary alert is not necessarily reportable.
Monitoring and ISO/IEC 42001
ISO/IEC 42001 can organise ownership, measurement, review and continuous improvement in an AI management system. ISO/IEC 23894 provides guidance for managing AI risks over the life cycle. These are supporting frameworks; neither turns general monitoring into the Article 72 plan or creates AI Act duties by itself.
Business example: prioritising applications
An organisation uses AI to support the ordering of applications, followed by a human decision. Months later validated accuracy falls, staff correct more recommendations, complaints rise and input distribution changes.
| Step | Illustrative action |
|---|---|
| Signal and metric | Compare accuracy KPI, complaint and override KRIs with the baseline |
| Threshold | Trigger review at the previously approved internal threshold |
| Investigation | Examine samples, inputs, labels, versions, instructions and affected groups |
| Risk and action | Reassess impact; increase human checks, fix data or settings, contact the provider or temporarily restrict a function |
| Evidence | Retain alert, analysis, decision, owners, changes and follow-up verification |
The organisation separately checks its legal role and high-risk classification; not every prioritisation tool falls under Article 72.
What evidence to keep
Depending on context, preserve dated metrics and dashboards, samples, logs, thresholds, alerts, complaints, human interventions, drift analysis, versions, provider communications, risk reassessments and corrective actions. Specific legal records depend on role and system category; this is a good-practice suggestion, not a universal statutory file. Tie each decision to an owner and follow-up result as part of AI evidence.
Review frequency and ownership
Monitoring may be continuous, weekly, monthly, quarterly or event-triggered depending on criticality, volume, impact, rate of change and incidents. These intervals are not presented as uniform legal deadlines. Business, Data/ML, IT, security, Risk, Compliance, Legal, a DPO where appropriate, audit and the provider may all contribute. Define who measures, interprets, escalates, decides and records.
How to implement monitoring
1. Identify the system, purpose, provider, legal role and applicable requirements.
2. Set proportionate risks, metrics, data sources, baseline and thresholds.
3. Assign owners, access to records, alerts and responses.
4. Test detection with representative cases and review findings with process experts.
5. Document decisions, changes and reassessments; revise monitoring when the system changes.
Conclusion
Monitoring turns real-world operation into decisions that can be checked. Organisations can use this cycle for quality and risk; providers of high-risk systems must also examine their specific Article 72 system and plan. Establishing roles, classification and timing first keeps good practice distinct from applicable legal duties.
Can you detect when your AI system changes?
Connect metrics, risks, owners and evidence to decisions after deployment.
Start the diagnostic →
Céntrika’s diagnostic helps you assess your starting point.