Design an evidence record for every consequential AI output.

Start with the question an incident will create

Six months after a client report is issued, someone disputes a figure. The useful record should show which documents were available, which calculation ran, what the model received, and who approved the final version.

A model name and a timestamp cannot answer that question. The evidence has to span the application, retrieval layer, inference service, tools, and review step.

This is an engineering proposal for that record, not a universal compliance template.

Record an execution manifest

Give each workflow run a stable identifier. Record its application release, prompt-template revision, model identifier, tokenizer and chat-template revisions, generation settings, and timestamps.

For self-hosted models, record verified artifact hashes and the runtime build. For hosted models, retain the exact returned model identifier and provider request ID where available. Distinguish a pinned version from a moving alias.

Capture source-document versions, retrieval configuration, tool names and versions, validated arguments, and results needed to explain the decision. Link the final artifact and the human review event to the same run.

Store sensitive evidence with access control and retention rules. A hash can help detect changes to a retained artifact; it cannot recover the original artifact or prove that an untrusted process used it.

Define replay before promising reproducibility

Replay may mean reconstructing the same inputs, obtaining an equivalent result, or reproducing identical output bytes. State which level the workflow needs. Each requires different retained artifacts and tests.

Temperature zero is not a general guarantee of identical output. Thinking Machines Lab documents how batch-dependent numerical kernels can change results as serving load changes, even when sampling is not the source of variation.

For exact replay, test the pinned hardware/software configuration under the relevant loads. Retain weights, tokenizer, templates, dependencies, tool responses, and any source data required by policy.

A hosted retired model may not remain callable.

For many workflows, the original approved output and its evidence are the essential record. Regeneration can support investigation, but it should not replace retaining the authoritative result.

Keep calculations and presentation connected

Have a deterministic tool calculate totals, ratios, or dates from validated inputs. Record the formula version, units, rounding policy, and source values. Deterministic code can still implement the wrong formula, so it needs its own tests.

Prefer rendering important figures directly from structured tool results. If a model writes them into prose, validate the relevant fields and units against their source before release.

String matching alone can miss a wrong denominator or interpretation.

An illustrative control record could link amount = 1243567, currency = USD, calculation version, and source IDs to the displayed figure. Validate the mapping as well as the digits. Escalate unexplained mismatches to review.

Separate regulatory facts from the proposed controls

The Federal Reserve's April 17, 2026 letter states that SR 26-2 replaces SR 11-7.

Footnote 3 of its attachment excludes generative and agentic AI from that guidance’s scope while pointing banks to their risk-management and governance practices.

That scope statement does not remove obligations under other applicable rules. FINRA Notice 24-09 reminds member firms that existing obligations continue to apply when they use generative AI.

The execution manifest above is a technical design recommendation. Its retention period, required content, and review process should be mapped to the organization’s actual obligations by the responsible compliance and legal teams.

Run an evidence-recovery exercise

Pick a completed synthetic workflow and ask someone other than its builder to recover the approved result, relevant inputs, calculation record, model identity, and review decision. Record what cannot be reconstructed.

Repeat after a model upgrade and a backup restore. If the evidence only works while the original developer remembers the configuration, the workflow is not yet auditable in practice.