AI Audit Exchange
Go back

Guide · By the Ingenious AI team · Published 14 November 2026 · Reviewed 14 November 2026

Common AI Audit Evidence Mistakes (and Their Fixes)

What is evidence mistakes?

Evidence mistakes are the recurrent handling errors that undermine an audit record's credibility on AI Audit Exchange: provenance breaks (edited or misrouted artifacts), completeness breaks (selection, mixed periods, silent gaps), preservation breaks (default retention, missing holds) and channel breaks (evidence shared outside scoped grants). Each is detectable from the room's own surfaces — manifest, seals, validation, completeness view and thread — which is what makes them fixable.

Mistake one: the helpful edit

Someone opens a suite export, fixes a typo, tidies a column, merges two parts 'for clarity' — and files the result. The artifact is now a derivative wearing a product's badge, and its seal certifies the edited file, not the product's record. One discovered edit re-prices every import in the room. The fix is absolute and simple: imports travel untouched, always; errors are corrected in the producing product and re-exported as new artifacts, with supersession noted in the thread. Helpfulness belongs in commentary, never in evidence. If your team has a 'prepare the files for the room' step, abolish it — that step is where this mistake lives.

Mistake two: the curated population

Filing the decisions that present well and omitting the awkward tail feels like advocacy; it functions as sabotage. Sampling mathematics turns on complete populations, and an examiner who discovers selection — they carry detectors for it in the form of volume sense, sequence gaps and cross-source checks — must discount everything. The fix: full runs, monthly, for every source, with awkward artifacts answered in the thread rather than absent from the manifest. An awkward record with an honest explanation is evidence; its absence is a conclusion about the filer. The decision layer states the rule at full length in the ReviewDesk import guide.

Mistake three: one room for everything

A single room accumulating multiple periods and both regimes feels economical and reads as a swamp: no clean manifest per audit, no per-period retention logic, completeness that cannot be counted, grants that expose more than any one engagement needs. The fix is the platform's organising rule — one room per regime per period — which costs nothing but the decision to start. Rooms are the unit of audit; keep the store at the audit's grain and every other discipline (freshness by period, retention by cohort, access by engagement) becomes simple arithmetic instead of interpretation. The period pattern is laid out in organising evidence by period.

Mistake four: the default decade

The engagement completes, the team exhales, and the room sits at the 12-month retention default while everyone believes the record is archived for the framework's 10-year horizon. Thirteen months later the room is gone and the belief is unfalsifiable. This is the most expensive mistake on the list because it is silent and total. The fixes are layered: set retention at creation, gate close-out on the completeness view (whose retention step stays undone at the default), read retention_months aloud from the close-out manifest, and sweep the shelf annually. A control that appears in four rituals cannot be forgotten by one busy person.

Mistake five: evidence by email

The brief went out by email, so evidence starts flowing by email 'just for this one item' — and the engagement's real record migrates to inboxes: unsealed copies, unbounded forwarding, no access log, no revocation at the end, and a successor auditor who inherits none of it. The fix is channel discipline with a structural spine: evidence moves only through the room — imports, uploads, grants, thread. Requests arriving by email get answered by an invitation into the room's process, not by an attachment. The one-item exception is how channel discipline dies; the exception is always one item, repeatedly, until it is the system.

Mistakes six and seven: silence and cosmetics

Silent gaps: a missing month, an absent source, an interrupted process — unmentioned until the auditor's sampling finds the shape of the silence. Auditors forgive gaps; they do not forgive discovering them. The fix is contemporaneous narration: note interruptions in the thread when they occur, in two lines, and convert tomorrow's finding into today's documentation. Cosmetic backfill: reconstructing the past to look continuous — regenerated exports filed as though original, summaries wearing product badges, chronologies composed in December for the whole year. The fix is the room's founding honesty: file what exists, badge what it is, date what you did. Every cosmetic improvement purchases a small presentational gain with the room's entire credibility as collateral.

Five actions to take

  1. Abolish any 'prepare files for the room' step: imports travel untouched; corrections happen at source and re-enter as new artifacts.
  2. File full runs for every source, every month — awkward artifacts get thread explanations, not omissions.
  3. Enforce one room per regime per period, without a miscellaneous room anywhere in the shelf.
  4. Gate every close-out on retention: set beyond default, read aloud from the manifest, swept annually.
  5. Refuse channel exceptions: evidence moves only through rooms, grants and the thread — never by attachment.

Worked example

A fictional health-tech firm, Canyon Vista Health Systems, commissions a pre-audit review of its room and receives a greatest-hits list: 2 edited imports (a merged export, a 'tidied' spreadsheet), a decision population with a visible selection seam (one decision family absent for a quarter), retention unset on the room, and a parallel email thread where the prospective auditor had already received 6 attachments. Five of this guide's seven mistakes, in one 58-artifact room.

The remediation took three weeks and no heroics: the edited artifacts were superseded by clean re-exports with thread notes; the missing family's exports were imported for the full quarter; retention was set to 120 months and read off the new manifest in the review meeting; the email channel was closed with a polite redirection into the engagement flow. The auditor's eventual report noted the corrections as transparency positives. Mistakes are common because the fixes are boring — and the fixes work because the room makes them visible.

Frequently asked questions

We already edited some imports. Is the room ruined?

No — supersede them honestly: re-export cleanly from the product, import as new artifacts, note the supersession and the reason in the thread. A corrected record with a visible repair beats a pristine story nobody can verify.

How would an auditor even detect selection?

Volume sense, sequence gaps, cross-source corroboration and rhythm breaks in import dates. Selection is legible to professionals; assume it will be seen and file completely instead.

Is one email attachment really a problem?

As an act, trivially small. As a precedent, it is how channel discipline dies — the engagement's record migrates to the least controlled channel one exception at a time. Redirect the request into the room; it takes a sentence.

Which mistake is most dangerous?

Default retention, because it is silent and total: every other mistake leaves a weakened record, but expiry leaves no record. Gate close-outs on the retention step and sweep the shelf annually.

Can contemporaneous notes really neutralise gaps?

Largely, yes. A two-line thread note written when an interruption happened converts a discovery into documentation. Auditors price the difference between a gap they find and a gap you filed — it is the difference between concealment and process.

Keep reading

All AI Audit Exchange guides · Auditor directory · About AI Audit Exchange · Security · Pricing · AI Audit Exchange home · Previous: How to Answer Auditor Evidence Requests Faster · Next: AI Audit Evidence for Small Companies: The Minimum Viable Discipline

How to Keep Audit Evidence for Ten Years (Practically) · One Evidence Room for Each Audit Period: The Rule and Its Reasons · What Auditors Look For in an Evidence Room