Move right only when evidence requires it
- 1 · Rules
Use deterministic logic for deterministic work
If known inputs lead to a known outcome, ordinary validation, templates and workflow rules are easier to test and operate than a model. Keep this baseline even when AI helps elsewhere.
- 2 · RAG
Retrieve current or private knowledge
Retrieval-augmented generation combines a generator with retrieved documents so an answer can use knowledge outside the model parameters. Its value depends on whether the right evidence is found and supplied.[1][2]
- 3 · Fine-tune
Train repeated behaviour, not a document library
Fine-tuning uses examples to shape task behaviour, format or style. It adds a dataset and evaluation lifecycle, so define a measured gap before training and keep changing factual knowledge outside the weights.[3]
Start with the acceptance test
Write a small set of representative inputs and acceptable outputs before selecting the technique. Include ambiguous cases, missing data and the failures that would matter to the business.[2][3]
A solution is ready when it meets that test within an acceptable cost, latency and review burden—not when it uses the newest architecture.[2][3]
Rules are an asset, not a primitive embarrassment
Validation, routing, permissions, calculations and mandatory wording usually need deterministic behaviour. Rules make the decision visible and give tests an exact result.[2]
Use a model inside the bounded step where variation is useful, such as classifying an enquiry or drafting a response. Keep policy and side-effect gates outside it.[2]
RAG is a search system before it is an answer system
Separate retrieval quality from answer quality. Check whether the expected source appears in the retrieved set before changing prompts or models.[1][2]
Preserve source identity and access controls through indexing and retrieval. A persuasive answer cannot repair missing, stale or unauthorised evidence.[1][2]
- Measure retrieval recall on representative questions
- Show sources to the reviewer
- Respect document-level permissions
- Define behaviour when no suitable evidence is found
Fine-tune only after the gap repeats
Consider fine-tuning when many examples show a stable behavioural gap that prompting and retrieval do not fix. Hold out evaluation examples so training success is not judged on material the model already saw.[3]
Plan dataset ownership, redaction, versioning, rollback and re-evaluation. If the need is simply updated facts, retrieve them instead of repeatedly training them into the model.[3]