There is a question that enterprise AI deployments almost universally cannot answer, and that regulators are beginning to ask with increasing frequency: show me exactly what happened in that decision, and prove it. Not approximately. Not probably. Exactly. With proof that the record has not been altered, that the decision would be identical if re-derived today, and that the verification does not require trusting the system that produced the original record.
Most current AI deployments cannot answer this question. They have logs. Logs are not replay. They have audit trails. Audit trails are not replay. They have model output records. Model output records are not replay. Replay — in the cryptographic and deterministic sense — means something specific: given a signed record of the input and the governance rule set at the time of the decision, any party can independently derive the identical verdict without calling any live system. The verification is local. The trust required is zero beyond the cryptographic primitives.
What the Regulations Actually Require
The regulatory landscape for AI is converging on traceability requirements that most current systems cannot satisfy.
EU AI Act, Article 9 requires that high-risk AI systems implement risk management systems that include "appropriate risk management measures" and that these measures be "systematic" and documented. More pointedly, Article 12 requires "automatic logging of events" sufficient to enable "monitoring of the operation of the high-risk AI system." The logging must enable "identification of any risks to health or safety, fundamental rights, or other societal risks." That is not an output log requirement — it is a decision traceability requirement.
Article 14 adds human oversight requirements, specifying that high-risk AI systems be designed such that "natural persons" can "understand the capacities and limitations of the high-risk AI system." Understanding capacities and limitations requires the ability to examine past decisions — which requires replay capability, not just logs.
SR 11-7, the Federal Reserve's model risk management guidance, predates modern AI but is increasingly applied to AI models in financial decision-making. It requires that models be "validated" — which means the model's behavior must be observable, reproducible, and auditable.
GDPR Article 22 adds another layer: where automated decisions have "significant effects" on individuals, those individuals have the right to "meaningful information about the logic involved." Meaningful information about the logic of a decision requires the ability to reconstruct that logic — which is replay.
Why Most Systems Cannot Replay
The structural reason that most AI systems cannot replay decisions is that their governance, to the extent it exists, is implemented probabilistically. The model was asked whether the request was appropriate. The model said yes. Neither the question nor the answer can be mechanically re-derived, because the model that answered is no longer identical to the model that answered then. Weights may have changed. Infrastructure may have changed. Prompt templates may have changed.
Even systems that use deterministic rule sets often fail at replay because:
- Rule versions are not recorded with decisions. Which version of the policy was in effect?
- Input canonicalization is inconsistent. The "same" input in different formats hashes differently.
- Audit records are not chained. A record can be altered without detection because there is no previous-hash linking.
- Signatures are absent or use shared infrastructure keys that rotate without notice.
Each of these failures is individually fixable. Together, they represent a systemic absence of replay architecture.
What Replay Architecture Actually Looks Like
A replay-capable governance system has the following properties:
- Input canonicalization. Before any evaluation, the input is serialized into a canonical form — typically JCS (RFC 8785 JSON Canonicalization Scheme) — and hashed. The canonical hash becomes the decision's identity. Two requests with different surface representations but identical semantic content produce the same hash and the same verdict.
- Rule set versioning. The exact rule set used to evaluate a given input is identified by a hash of the compiled rule artifact. Every decision record contains this hash. Given the hash and a rule set archive, any party can recover the exact rules that applied.
- Verdict signing. The verdict — input hash, rule set hash, decision output, timestamp — is signed using HMAC-SHA256 or stronger. The signing key's version is recorded. The signature binds the verdict to the specific input and rule set at a specific time.
- Hash-chaining. Each decision record contains the hash of the previous record. A chain of decisions is tamper-evident: any alteration to any record in the chain invalidates all subsequent records. Gaps in the chain are detectable as missing records.
- Offline verification. An auditor with the signed record, the rule set archive, and the verification key can re-derive the verdict on a machine with no network access.
The Replay Guarantee in Practice
In practice, a governance system with full replay capability produces the following workflow for an audit inquiry:
- Auditor identifies the decision in question by session ID and turn number.
- Auditor retrieves the signed decision record from the audit log.
- Auditor verifies the hash chain integrity from the decision record back to a published root.
- Auditor downloads the rule set version referenced in the decision record.
- Auditor runs the local verification tool against the canonical input and rule set.
- Verification produces the same verdict as the original decision record.
Steps 4 and 5 require no access to any live system. The verification is purely local computation over signed artifacts. The infrastructure that produced the original record does not need to be running, intact, or trusted.
This is what enterprise AI audit looks like when replay is built into the architecture from the start. For most current AI deployments, the equivalent workflow involves reconstructing approximately what might have happened from partial logs, model version notes, and the system administrator's recollection.
The regulatory direction is clear. The technical solution is known. The organizations that will be prepared are those that treat replay not as a logging feature added after deployment, but as a first-class architectural requirement that shapes every other design decision. Replayability is not coming to enterprise AI governance. It is already required. Most systems just do not know it yet.