📖 Content moderation glossary
What is a confidence score in moderation?
In short
A confidence score is a value, often between 0 and 1, that indicates how certain an AI model is that a piece of content belongs to a category such as spam, insult or hate speech.
Moderation models rarely answer just yes or no. They output a score per category, and the system decides what to do with it: act automatically above a high score, send to a person in the middle, leave the content alone below a low score.
Example: a comment gets 0.97 for spam because it contains a known scam link, so it is hidden at once. Another gets 0.55 for harassment because it is a harsh reply in a heated thread; it goes to a moderator instead of being hidden automatically.
A confidence score is not a measure of truth: a model can be very confident and wrong, especially on content unlike what it has seen before. Scores are useful only with well-chosen thresholds, regular checks of the decisions and human review of the uncertain zone.