AI Engineering Controls

Controls are what make AI-assisted software feel boring in production. That is the goal. Excitement belongs in the lab.

Versioning

Prompts in git. Model names in config, not hardcoded strings scattered across files. Lock files or pinned API versions. Every deploy knows exactly which model and prompt it shipped with.

Evaluation

Golden datasets for extraction and classification. Score precision and recall before release. Track drift when vendors update models. A small set of real files beats a hundred synthetic edge cases nobody sees in production.

Approvals and logs

Human sign-off where SOP requires it. Structured logs: input hash, model version, output, latency, user override. Logs go to the same retention policy as the rest of your regulated data.

Retries and security

Retry transient failures with caps. Never retry a partial write without idempotency keys. Strip PII from logs. Restrict which services can call models. Review new tools and data paths like any other production change.