How it works

From content ingestion to automated action in one pipeline.

1. Ingestion

Content arrives via API, webhook, or social platform connector.

POST /v1/moderation Webhooks Facebook Instagram YouTube

2. AI Moderation

Smart routing sends content to the best provider. Standardized result returned.

OpenAI Azure HuggingFace LLMs

3. Rules & Actions

Rules engine evaluates results. Actions execute automatically.

Hide Delete Flag Webhook

4. Manual Review

Edge cases flagged for human moderators to approve or reject.

Review queue Approve Reject
Step 1

Content arrives from anywhere

Submit content through any channel. Use the direct API for programmatic access, configure webhooks to receive content from external platforms, or set up native social media connectors that ingest comments and posts automatically.

  • API: POST /v1/moderation for single or batch moderation
  • Webhooks: Generic endpoint receives and normalizes external payloads
  • Connectors: Native Facebook, Instagram, YouTube integration

Example: Facebook comment

json 
// Facebook sends comment via webhook
{
  "source": "facebook",
  "content_id": "comment_123",
  "author_id": "user_456",
  "text": "Comment text here...",
  "metadata": {"post_url": "..."}
}
Step 2

AI moderates the content

Content is routed to the best available provider based on language, content type, category requirements, and cost. Each provider returns standardized results with scores from 0.0 to 1.0 across 10 content categories.

  • Smart routing: Picks the best provider if not specified
  • BYOK: Use your own provider API keys (or use platform keys)
  • Standardized output: Same format regardless of provider behind the scenes

Standardized response

json 
{
  "model": "openai/omni-moderation-latest",
  "results": [{
    "category": "hate",
    "score": 0.02,
    "flagged": false
  }],
  "flagged": false,
  "usage": {
    "latency_ms": 243,
    "provider": "openai"
  }
}
Step 3

Rules enforce your policy

Define your moderation rules with three simple tools: forbidden words for instant blocking, category thresholds for automated decisions, and your community charter written in plain language for context-aware AI moderation.

Forbidden words block content instantly, before any AI call — zero cost

Category thresholds trigger automatic actions when scores exceed your limits

Your community charter guides context-aware AI decisions and sets the tone for your platform

Rule configuration

json 
// Configure your moderation thresholds
PUT /v1/settings

{
  "thresholds": {
    "hate":       { "accepted": 0.3, "rejected": 0.7 },
    "harassment": { "accepted": 0.3, "rejected": 0.7 },
    "sexual":     { "accepted": 0.2, "rejected": 0.8 },
    "violence":   { "accepted": 0.3, "rejected": 0.7 },
    "self-harm":  { "accepted": 0.2, "rejected": 0.8 },
    "spam":       { "accepted": 0.4, "rejected": 0.8 }
  },
  "actions": {
    "on_reject": ["hide", "flag_for_review"]
  }
}
Step 4

Review edge cases manually

A dedicated, intuitive workspace for your Trust & Safety teams. Review flagged content, manage user reports, and override AI decisions with full auditability.

  • GET /v1/contents — browse flagged content
  • Use the moderation console to approve or reject flagged items
  • Webhook notifications for new review items

Everything is logged

json 
// Complete audit trail for every content item
{
  "content_id": "...",
  "source": "webhook",
  "platform": "facebook",
  "status": "actioned",
  "moderation": {
    "provider": "openai",
    "flagged": true,
    "results": [...]
  },
  "actions": [{
    "rule": "Hide hate speech",
    "type": "hide",
    "status": "executed"
  }]
}

Action Engine

When content exceeds your thresholds, the Action Engine executes the right response automatically — on the platform itself.

Delete

Permanently remove violating content from your community. Carried out through the native platform API (Facebook, Instagram, YouTube).

Hide

Make violating content invisible to your users. Preserved for review and audit, but end users won't see it.

Flag for Review

Send borderline content to your team's manual review queue. Moderators approve or reject.

Allow

Whitelist specific content, authors, or channels so they are never flagged or removed.

Notify Webhook

Send real-time alerts to your team via webhook, enabling custom workflows or integrations.

Mark as Spam

Platform-specific spam classification. On YouTube, comments are flagged as spam to reduce visibility.

Ready to simplify your moderation stack?

Get a complete moderation platform with AI, rules, and a human review interface. No complex integrations required.

Try it for free