AI Agent Audit Trails: What to Log, and Why It Is No Longer Optional

A support agent built on a large language model reads an incoming complaint, looks up the customer in the CRM, checks the order history, decides the claim is valid, issues a refund and sends a reply. It does this several hundred times a week without a human reading any individual case. Six weeks later, a customer alleges the outcome was unfair, asks on what basis the decision was made, and copies in the regulator. The compliance team asks a reasonable question: show me exactly what happened.

In most organisations, what comes back is a chat transcript from a vendor console, retained for thirty days, showing the prompt and the reply. It does not show which customer records were read, which internal tool was called with which parameters, which version of the system prompt was live that week, whether a human approved anything, or why the agent chose refund over escalation. That gap is the practical problem. It is not a technology problem — the events happened and could have been recorded — it is a governance problem, and it is now the subject of explicit regulatory expectation.

Why a chat transcript is not an audit trail

Traditional model governance assumed a model that scores an input and returns an output. An agent is different in three ways that matter for evidence.

It takes actions with external effect — writing to systems, sending messages, moving money, changing records — so the harm is not a bad score but a completed act. It chains steps, meaning a single business outcome is the product of a sequence of tool calls, retrievals and intermediate reasoning, any one of which can be the point of failure. And it often carries memory or retrieved context, so the same input on two different days can produce different behaviour because the underlying knowledge base, prompt or model version changed underneath it.

Because of this, an audit trail for an agent has to record a sequence of events, not a single input-output pair. If you cannot answer "which of the twelve steps went wrong", you cannot fix the control, apportion responsibility, or defend the outcome.

What the rules actually expect

Several regimes converge on the same requirement from different directions, and none of them are satisfied by application logs alone.

The EU AI Act requires high-risk AI systems to be technically capable of automatically recording events over their lifetime, and places obligations on providers to retain those logs. Deployers of high-risk systems must keep the logs the system generates, to the extent they are under their control, for a period appropriate to the purpose — with a floor of at least six months unless other Union or national law requires longer. Penalties for breaching most obligations under the Act reach up to 3% of global turnover, rising to 7% for prohibited practices.

The UK and EU GDPR approach it through accountability: a controller must be able to demonstrate compliance, not merely assert it. Where an agent makes decisions with legal or similarly significant effects on individuals, the transparency and contestation provisions require you to explain the logic involved and to allow a person to challenge the outcome. A subject access request must be answered within one month. Maximum penalties reach 4% of global turnover.

ISO/IEC 42001, the AI management system standard, expects documented information to be retained as evidence that the system operated as designed, and expects records of impact assessments and operational controls. NIST's AI Risk Management Framework treats traceability as a precondition for measuring and managing risk at all. Sector supervisors add their own layer — model risk management expectations in financial services, clinical governance in health, statutory record-keeping in regulated professions.

The minimum record for a single agent action

Rather than logging everything, define one record structure and apply it to every consequential action. Most teams find the following list is the point at which reconstruction becomes possible.

• Trace identifier linking every step of one business outcome, so a refund can be followed end to end rather than assembled from timestamps.
• Initiating principal — which human, customer, schedule or upstream system triggered the run, and under whose authority.
• Agent identity and version, including the deployed configuration, not just the product name.
• Model and prompt versions pinned at the moment of execution. A prompt edited last Tuesday changes the meaning of every log written before it.
• Tool and system calls, with the parameters sent and the response received — this is where agents actually act.
• Data touched: which records, from which store, under which access credential. Keep references and identifiers rather than copying payloads.
• Output and disposition: what the agent decided, what it executed, and what it declined or escalated.
• Human intervention points: who reviewed, approved, overrode or ignored, and when.
• Guardrail events: policy blocks, refusals, retries, timeouts and failures — including the ones that worked, which are your evidence the control exists.
• Integrity marker such as an append-only write or hash chain, so the record cannot be quietly edited by the team that owns the agent.

Observability is not the same as an audit trail

Engineering teams frequently believe this is already solved because the platform emits traces. The two artefacts serve different masters and rarely satisfy each other.

Dimension	Engineering observability	Governance audit trail
Purpose	Diagnose failures and latency	Reconstruct and defend a decision
Coverage	Often sampled	Complete for consequential actions
Retention	Days to weeks, cost-driven	Set by law, contract and limitation periods
Mutability	Editable by the owning team	Append-only, segregated access
Content	Whatever is convenient to emit	A defined, reviewed record schema
Retrieval	Ad hoc queries by engineers	Findable by case reference, within a deadline

The retention trap: logging too little and too much

Agent logs are frequently personal data, and sometimes special category data, because they contain whatever the agent read. Building a comprehensive log store therefore creates a new processing activity that needs a lawful basis, a retention period, an entry in your record of processing, and inclusion in your data protection impact assessment. Organisations regularly create a shadow database of sensitive material inside a governance tool with weaker controls than the source system.

The workable compromise is to log references rather than payloads: record which record identifiers were accessed and under which credential, so the trail is reconstructable against the source of truth without duplicating the content. Where free-text must be retained, apply the same classification, access control and retention rules you would apply to the original. Then set retention deliberately: the AI Act floor, any sector requirement, and the limitation period for the claims the log would defend — whichever is longest — rather than whatever the vendor's default happens to be.

What this does not cover

An audit trail is evidence, not a control. It records that an agent acted; it does not prevent the action, remove bias from the underlying model, or make an unlawful process lawful. Organisations sometimes treat comprehensive logging as the AI governance programme, when it is one component alongside classification, testing, human oversight design and incident response.

Nor does a good trail settle whether a given system is high-risk under the EU AI Act, whether a decision falls within the automated decision-making provisions of data protection law, or how the Act's obligations phase in for your specific role as provider or deployer. Those are legal determinations that turn on facts about your use case, and the timing and detail of the obligations continue to be clarified through guidance and standards work. Where an agent operates in employment, credit, health, essential services, law enforcement or safety-critical contexts, or where you are contractually bound by a customer's own regulatory regime, take specialist legal and sector advice on classification and retention before you finalise the design. The reconstruction drill is still worth running on Monday; it will tell you what you can currently prove, which is the input to every one of those conversations.