Guide · Audit evidence

Audit evidence for AI model and provider decisions: what to keep

Sooner or later someone asks why a team pays for one AI provider rather than another. A good answer is a record another person can rerun and get the same result from, not a slide.

Keep the inputs, exactly

• The usage file the comparison was run on, as it was, not a cleaned copy.
• The mapping from its columns to the fields the analysis expects, including what counted as success.
• The price table, with where it came from and the date it applied to.
• The rules: minimum sample size, how ties were decided, what was excluded and why.

Fingerprint everything

Record a SHA-256 fingerprint of each input and of the report. Anyone holding the same files can recompute the fingerprints and confirm they have the same inputs. AIYVO’s analyzer writes these fingerprints, the fingerprint of the analyzer itself and the reference time into every report.

Fix the reference time

Analyses that depend on “now” change when they are rerun. Record the reference time used. With the same files and the same reference time, AIYVO’s analyzer produces byte-identical reports, so a rerun either matches or shows exactly what changed.

Keep what was withheld

A record of the comparisons that were not made matters as much as the ranking: models with too few records, figures withheld for missing costs or outcomes, orderings withheld because groups did different work. These show the decision did not rest on data that could not support it.

Know the limits

A fingerprint identifies a file. It does not show that the data in it is true or complete, and it is not a signature or a chain-of-custody record. The figures are only as good as the export they came from; customer-reported costs remain customer-reported until compared with invoices.