AI sounds confident. Is it right? How SMEs can assess AI answers

The quotation summary looks convincing: clear paragraphs, a brief recommendation and even source references. Only on a second reading do you notice that setup costs are missing. The lowest monthly charge has become the supposedly lowest total cost. AI has done useful writing work while quietly changing the basis of the decision.

This is a fictional example. The underlying question is real: how do we know whether an AI result is good enough for our work? “Sounds reasonable” is a weak standard, particularly when the fluent wording makes you want to use the answer immediately.

A Swiss SME does not need a complicated testing platform first. It needs verifiable expectations for a specific task. Our recommendation is to assess quality where the output will be used. A useful summary, an accurate customer reply and a good writing draft have different success criteria.

NIST's Generative AI Profile describes confidently presented false content as confabulation, often called hallucination. It also recommends checking sources and citations in AI output. The profile is a voluntary risk framework, not a Swiss certification.

Source: NIST AI 600-1, July 2024, particularly section 2.2 and MS-2.5-003.

In everyday work, a broader view helps. An answer need not invent anything to be unusable. It can omit an exception, give the wrong weight to two valid documents or accurately repeat an old rule that has been replaced. Even a correct number can answer the wrong question.

Do more than look for obvious mistakes. Ask which decision-relevant information is missing and which assumption has slipped in unnoticed. Checking starts with the assignment. “Summarise this” does not specify what must never be lost.

For answers based on supplied sources, Microsoft distinguishes dimensions including groundedness, relevance and completeness. Groundedness concerns support in the context; relevance concerns the question asked; completeness compares the answer with expected information. The documentation checked on 23 October 2026 still labels some evaluation methods as Preview.

Source: Microsoft: RAG evaluators. RAG here means an application that retrieves relevant information from a source collection before answering.

The table translates those ideas into everyday checks. It is our working aid, not an official scoring scale.

An answer can succeed in one column and fail in another. That is why “eight out of ten” alone offers little guidance. Record the actual defect. Only then can you see whether to improve the instructions, sources or processing.

A link at the end of a sentence is not a completed fact check. Open the source and find the passage that is supposed to support the claim. Does the time period match? Does it cover the right product, organisation or contract version? Is it a binding rule, an available option or merely an announcement?

For internal documents, we recommend asking for more than a filename. Request the section and version or date where available. If documents contradict each other, the assistant should show the conflict rather than silently resolve it through a persuasive guess.

Grounding has a limit too: an answer can closely follow an outdated template and still be wrong for today. Deciding which document takes precedence belongs to knowledge management. A better prompt cannot fix unclear ownership.

For a customer-facing claim, we would ask the reviewer: can you show where this sentence comes from and why that source applies here? If the answer points only to the AI, the review is unfinished.

Our suggested starting point is around 20 carefully chosen cases from one clearly defined workflow. That is a manageable initial size, not statistical proof or a universal safety threshold. One successful demonstration is insufficient; a huge, neglected question bank is not much better.

Include typical requests, difficult exceptions and cases where the available documents do not support an adequate answer. Use approved, anonymised or synthetic test data. Testing should represent the work without unnecessarily distributing confidential information to more systems.

Write down expected key points before seeing the AI response. A quotation comparison might require total costs, contract duration and explicitly excluded services. For an internal knowledge question, the correct outcome might instead be that responsibility cannot be established from the available sources.

Do not judge whether the AI produces the exact wording of your reference answer. Assess the substantive requirements. Otherwise, you may penalise good alternative phrasing while overlooking a neatly copied but factually incorrect statement.

Suppose someone asks for the “current price list”. The sources contain two lists for different customer groups. A useful response might first clarify the customer group. Counting every follow-up question as failure can reward precisely the behaviour that later causes trouble: confident guessing.

For each case, specify whether you expect a direct answer, clarification or justified uncertainty. Restraint is a quality feature when the evidence is incomplete. For a simple, clearly supported question, however, the assistant should not hide behind endless generic caveats.

Good AI quality does not mean always producing an answer. It means giving the right kind of response for the evidence available.

Check subsequent conversation turns as well. Does a qualification survive a request for a shorter version? Does “expected” suddenly become a promise? A useful test need not end after the first answer. It should represent the way people actually work with the tool.

An awkward phrase and an invented delivery commitment do not belong in the same error bucket. Decide before testing which failures rule out using the output. In a customer reply, that might be an unsupported promise. In a summary, it might be omission of an essential condition.

A wider tolerance makes sense for internal brainstorming. A person selects and rewrites the suggestions. If the output informs a purchase, stronger substantive checks are needed. We recommend matching review effort to the possible consequences.

Alongside the result, record an error class: wording correction, substantive rework or unusable. This deliberately simple classification says more about the remaining work than a long decimal score with an appearance of precision.

A critical case must not disappear inside a good average. If an application invents a material contract condition, “the other 19 answers were good” does not help that decision. Investigate the cause and restrict use until that failure mode is appropriately addressed.

Automated assessment can help with large numbers of answers. But Microsoft's discussion of evaluators explicitly includes testing the evaluators themselves, using controlled defects and known expected outcomes. A plausibly explained score is not automatically a reliable measurement.

Source: Microsoft: Who evaluates the evaluators?

Our practical conclusion is to calibrate an automated judge against examples reviewed by subject specialists. Does it detect a missing exception? Does it accept correct answers? Does it confuse length with quality? Check its verdict rather than merely admiring the explanation.

An initial manual review with the responsible subject experts is usually enough to get started. As volume grows, technology can speed up screening. Keep substantive accountability visible. “The other model approved it” is too weak a basis for a meaningful business decision.

Quality and value belong together. Measure more than how quickly a draft appears. Include time spent opening sources, correcting and approving it. If three minutes of writing becomes ten minutes of checking, the economics differ from the demonstration.

Compare the same workflow with and without AI. Use comparable tasks and record who reviewed them. With small samples, this provides direction rather than a reliable company-wide projection. It should still be an honest comparison.

First identify the bottleneck. AI might save substantial time on standard letters while creating too much rework on complex contract comparisons. Narrower deployment is then a sensible product decision. A tool need not be approved for every task to be worthwhile.

A revised prompt, a new source or a different model version can affect behaviour. Keep a fixed core of test cases. Repeat them after material changes and add new failures observed in use. We recommend a small, maintained test set rather than a one-off acceptance exercise nobody revisits.

Record the date, application, visible model name where available, sources and result. If a product does not disclose its internal components, document that limitation. Do not invent version certainty you cannot observe.

Keep development and acceptance distinct. Repeatedly tuning a prompt for the same 20 cases can improve its apparent performance without improving unfamiliar tasks. Hold back some additional cases for an independent check. This does not replace broader assessment for consequential applications, but it makes the initial decision more traceable.

One concrete step is enough for this week: choose a recurring task, agree its indispensable content and assess a small set of typical and difficult cases. Have the responsible subject expert judge the results. Then decide where AI already helps and where a clear boundary remains necessary.

Our position is optimistic but demanding. Good AI is about more than polished language. It should produce usable results in its intended setting, expose uncertainty and allow sensible checking. That is how an impressive answer becomes a dependable working tool.