Auditing AI-generated code: what a diligence record actually contains

Why the usual artifacts don't audit correctness

Three things get offered as evidence. None of them checks whether the code is right.

Provenance tells you where code came from, not whether it behaves. The provenance standards say as much themselves — attestation covers the supply chain, signatures, build integrity. It is silent on correctness by design.

Test coverage tells you that tests ran and lines executed. It does not tell you the tests check the right thing. When the tests and the code are written from the same intent — increasingly, by the same model, in the same pass — they encode the same assumptions. A test suite can be green because the code is correct, or because the test agrees with the same mistake. Coverage cannot tell those two apart.

An LLM code review reads the code and comments on it. But a reviewer that derives its opinion from the code inherits the code's blind spot. If the model misread the intent while writing, it will misread it the same way while reviewing. This is AI checking AI. It is not an independent check.

The common failure underneath all three has a name: intent drift — code that diverges from what was meant while passing every test written alongside it. One 2025 study measured this across three frontier models: asked whether code met its specification, they judged conformance unreliably to begin with — and requiring them to also explain the verdict and propose fixes made it sharply worse, in one case collapsing from 52% to 11%. The extra reasoning did not sharpen them; they assumed flaws that were not there and rejected correct code (Jin & Chen, ASE 2025). The more an LLM is asked to reason like a reviewer, the less its verdict can be trusted.

There is a fix implied in that failure: the verdict cannot be the model's to give. In what follows a model is used once — to propose the standard — and never to sit in judgment; the judging runs on CPU against a frozen artifact, where no reasoning remains to destabilize. Freezing does not make the proposal right — a frozen misreading is reproducibly wrong. What freezing buys is different: the standard becomes a readable artifact an auditor can attack directly, and the derivation is repeated across runs, with instability reported as underspecification rather than averaged away. Determinism is a claim about re-litigation, never about truth.

What independence actually requires

An independent check needs a reference the code cannot have contaminated. The reference has to come from the intent, derived blind to the code body.

That is the whole mechanism, stated plainly. From a function's signature and its docstring — the stated intent — a standard is derived without reading the body. The code is then fired against that standard. Because the standard never saw the implementation, it does not share the implementation's blind spot. A human's own tests can only find bugs the human thought to look for. A standard derived from intent, blind to the code, is not bounded that way.

The independence is scoped precisely — to the body, not the intent. And the intent has an origin that matters just as much. An intent a person wrote is a genuinely separate source. An intent a model co-generated next to the body, which a person then read and approved, is not the same thing: ratification proves accountability, not independence — a shared misreading can survive a nod. agetra does not blur the two. The record carries the intent's provenance — authored, ratified, or unknown — and the strength of the independence claim follows that field. A standard is only as independent as the intent behind it; the record states which kind was supplied instead of assuming the best one.

Two properties make the result auditable rather than just another opinion:

• The body is never read. The standard is projected from the interface — signature and docstring — only. Your code stays local. What crosses is a body-free description of intent, nothing more.
• The verdict is deterministic. Deriving the standard is the one step where a model is involved, and it happens once; the standard is then frozen. The verdict — does the code meet the frozen standard? — runs on CPU against the frozen artifacts, with no model in the loop. Re-running it reproduces it exactly. There is nothing non-deterministic to re-litigate.

The anatomy of a diligence record

A record is auditable when you can enumerate what it contains and re-check each part. This is what a diligence record holds:

• The intent it was checked against — the exact signature and docstring, frozen and versioned, with its provenance: authored by a person, ratified from a generated draft, or unknown. If the intent changes, that is a new record, not a silent update.
• The standard — the oracle derived from that intent, frozen. You can inspect what the code was actually held to.
• The verdict, per behavioral clause — one of three states: conforming, diverging, or can't tell. Not a score. Where the check has teeth, it says so; where it cannot decide, it says that, too.
• The explored input space — the reproduction inputs, including the version of any standard data used (time zones, numeric standards). Reproduction means re-running the same exploration, not trusting a summary.
• What it does not cover — out-of-scope surfaces and anything explicitly waived, named. A record that cannot show its own limits is not evidence; it is decoration. There is no false 100%-green.
• Provenance of the act — when it ran, who ratified the intent, and the fact that it reproduces from CPU alone, with no model re-call.

Schematically, the fields an auditor checks:

intent:        <hash of frozen signature + docstring>
  provenance:  authored | ratified | unknown
standard:      <hash of derived oracle>
verdict:
  - clause: <behavioral clause>   state: conforming | diverging | can't-tell
inputs:        <explored space + standard-data version>
out-of-scope:  <named surfaces>   waived: <named, with reason>
ratified-by:   <identity that confirmed the intent>
reproduces:    cpu-replay, no model

What it is — and what it isn't

A record finds divergences in the explored input space. It does not claim their absence. That distinction is the honest core of it: the standard being met is diligence, not perfection. A signed run is evidence that you looked, carefully, in a way another party can reproduce — not a proof that nothing was missed.

So: it is a diligence record, contemporaneous and attributable and reproducible. It is not a certificate of correctness. It is not a bug-finder. And it is not another model's opinion about the code — the whole point is that the standard was derived without seeing it.

Stated exactly, the claim is three narrow things and no wider: independent of the body, not of your intent; diligence, not correctness; reproducible, not infallible. Each of the three is something a second party can check. Nothing broader is being claimed.

Why this matters now

From 9 December 2026 the EU Product Liability Directive treats software as a product, and on sufficiently complex systems the burden of proof can shift toward the maker. "We were careful" becomes something you have to show, not say. And showing it is easiest when the evidence was captured as the code was written — contemporaneously, in the loop where it is generated — not reconstructed under subpoena months later. How a diligence record maps to that — and what "legally defensible" does and does not mean — is its own subject, Legally Defensible Code (a companion dossier, in preparation).

Scope & status

agetra is pre-launch. The first version verifies pure, deterministic Python logic — the place where "does it do what was meant?" has a clean answer. If you audit or ship AI-generated code and this is the evidence you wish you had, join the waitlist. The body of your code never leaves your machine.