Industry
Building the audit trail in from the first commit
In regulated work the question is never just whether the system was right. It is whether you can show why it decided what it decided, a year later, to someone hostile.
Financial institutions do not fail exams because a model was wrong. They fail because nobody could reconstruct what the model saw, what it returned, and who acted on it.
Log the inputs, not just the outputs
Storing a decision is easy and nearly useless on its own. What an examiner wants is the state of the world at decision time: the document version, the retrieved passages, the prompt, the model identifier, the thresholds in force that day. Store all of it, immutably, keyed to the case.
Version everything that can change an answer
Prompts change. Retrieval indexes change. Models get deprecated and swapped. Each of those is a code change in our repos, reviewed and tagged, so a decision made in March can be replayed against exactly the March configuration.
Design for the exception path first
The interesting cases are the ones the system declines to handle. Route them to a named human, record the override, and treat a rising override rate as the primary health metric for the whole system. It will tell you about drift long before your evaluation suite does.
The trade you are actually making
Auditability costs latency and storage. It does not cost accuracy. Teams that skip it are not buying performance; they are borrowing against an exam they have not scheduled yet.