How to Show AI Confidence Without Faking Precision
Showing AI confidence in your UX builds trust, but a fake 94% score destroys it. How to signal model uncertainty honestly without inventing numbers you cannot defend.
Never show a confidence number you cannot defend. A model that prints "94% confident" next to an extracted invoice total looks precise and is usually lying, because the number is a softmax artifact, not a calibrated probability of being right. Users trust it anyway. Then it is wrong at 94 percent and you have burned more trust than if you had shown nothing. The right move in AI-native UX is to signal uncertainty honestly, which almost never means a two-decimal percentage.
The thesis: confidence display is a trust feature, and a fake precise number is an anti-trust feature. Show doubt where doubt exists, and show it in a form the user can act on.
Why raw model confidence scores mislead
The probability a language model assigns to its own output is not the probability that the output is correct. Those are different quantities, and they diverge hard. Models are often most fluent and confident exactly when they hallucinate. So the "confidence" you pull off the logits tells you how sure the model is about the next token, not whether the extracted number matches the document.
Slapping that number in the UI launders a guess into an authority. The user reads "94%" and stops checking. That is the opposite of what you wanted. If you are going to make claims about reliability in the interface, they have to be claims you can stand behind, which is the same discipline as making AI decisions defensible after the fact.
How to signal uncertainty honestly
Replace the fake number with signals tied to something real.
Use bands, not decimals. "High / needs review / uncertain" maps to thresholds you actually calibrate against your eval set. Three buckets you can defend beat a percentage you cannot.
Tie confidence to evidence. Instead of "94%," show the source: the highlighted span in the document the value came from, the transaction the category was inferred from, the sentences the summary drew on. Evidence lets the user verify in two seconds. That beats any score. It is why grounding output in a citation is the strongest trust signal an AI-native product has.
Escalate the low-confidence cases, do not just label them. If the model is unsure, route that item into a human review queue instead of shipping it with a yellow badge and hoping someone notices.
Disagree with yourself out loud when you should. If two extraction methods conflict, say so and show both. Manufactured certainty is the failure mode. Honest conflict is a feature.
When to show confidence at all
Not every output needs a confidence signal. Showing one everywhere trains users to ignore all of them, the same way a car that always beeps teaches you to tune out the beep.
Show it when the cost of a silent error is high: money, legal, medical, anything that flows downstream unchecked. In AI-native case management or bookkeeping, a wrong value that no one flags becomes a wrong filing. There, uncertainty signals earn their space.
Skip it when the user is already reviewing the output anyway, or when the action is trivially reversible. A draft email the user reads before sending does not need a confidence badge. The user is the review step.
The rule I hold products to
If you display a confidence number, you must be able to answer two questions: what does this number mean, and did you check that it is calibrated. If the answer is "it is whatever the model returned," take it out. An unlabeled, uncalibrated score is decoration pretending to be information.
We build our extraction surfaces at Ficary to show the source line, not a spurious percentage, because an accountant trusts a highlighted document a lot more than a number the software made up. Honest uncertainty, tied to evidence, with escalation for the hard cases. That is confidence display that survives contact with a real user. The fake 94 percent does not.