← Back to Projects
AI-MOD

AI-MOD: Multi-Platform AI Moderator

AI-MOD is an autonomous Twitch & YouTube live chat moderator. Open-source, self-hosted, and fully customizable. Swap your LLM preference (Ornith, GPT-4o, Claude 3.5) via config — no vendor lock-in. Ingests live chat, evaluates against platform community policies + your channel's unique rules, and executes timeouts, message deletions, warnings, or bans silently via MCP tools.

Built for Every Streamer — Tailor-Made & Fully Customizable

This isn't a one-size-fits-all SaaS bot. Every streamer gets their own isolated instance of AI-MOD, giving you complete code ownership and absolute data privacy. You specialize the mod to your channel's unique rules and culture using plain English, database parameters, or prompt tuning.

Per-Channel Rules (Vector Matching)

Write custom rules in plain English. We generate 384-dimensional vector embeddings via HuggingFace and store them in PostgreSQL. AI-MOD matches incoming chat messages semantically in real-time, catching slang and spelling variations.

rules = 'No backseating, delete game spoilers'

Bring Your Own LLM

Choose the LLM plan you are already comfortable with. Run a local 9B model like Ornith for free, or connect to OpenAI or Anthropic using existing plans. Switch models with zero code changes.

OLLAMA_MODEL=gpt-4o

Custom Action Tiers

Map violations to standard Twitch/YouTube moderation actions. Delete specific messages for minor slip-ups, send private whispers for warnings, enforce timeouts for moderate rule breaking, or issue permanent bans.

actions: ignore | whisper | delete | timeout | ban

Action Thresholds & Prompts

Fine-tune moderation severity. Edit the prompt in prompt/templates.py to favor warning first-time offenders, adjust timeout durations, or ignore specific inside jokes entirely.

default_timeout = 600

Trusted Users Bypass

Skip evaluations entirely for specific users (streamers, regular moderators, friendly bots). They bypass the LLM and DB check, ensuring zero false-positive flags.

TRUSTED_USERS=streamer,moderator_bot

Isolated Multi-Channel Hosting

Run a single bot instance to moderate multiple channels. Each channel has its own independent rules, timeout configurations, auto-mod toggles, and logs isolated in PostgreSQL.

TWITCH_CHANNELS=chan1,chan2

Interactive Engagement (Polls & Predictions)

Manage stream engagement dynamically. Trigger channel points betting and create or end polls via MCP tools (create_poll, create_prediction) if authorized via OAuth scopes.

scopes: channel:manage:polls | channel:manage:predictions

Hands-Free Voice Control & Drive Backup

Enable the mic overlay in OBS. Saying "clip that [title]" triggers a clip on Twitch with that title, then backups the video clip directly to your mapped Google Drive folder.

triggers: "clip that" | "assistant"

Hardened Security Architecture — Built for Ultimate Trust

AI-MOD is designed from the ground up to protect your stream, your credentials, and your channel’s integrity. We implement strict defense-in-depth security gates across the entire application stack.

🛡️ Zero-Trust API Guard

All control endpoints are locked behind UUID token authentication (`X-API-Key`) with strict Origin/Referer verification. Disables wildcard credentials to prevent cross-origin (CSRF) hijacks.

🛡️ Prompt Injection Shields

Scans incoming chat logs and streamer console inputs against regex-based prompt injection patterns. Enforces a strict server-side whitelist for allowed LLM execution actions.

🛡️ Container & DB Isolation

Runs inside Docker as a non-privileged `appuser` (non-root). Restricts the database and memory stores (Redis, Postgres) to password authentication on an isolated internal bridge network.

🛡️ Anti-Replay Voice Logic

Protects hands-free voice controls by deduplicating audio command hashes within a sliding 30-second window, and auto-enforces a rate cooldown to block microphone replay attacks.

How AI-MOD Works: Interactive Data Flow

AI-MOD operates on two separate AI pipelines. Choose a pathway below and click on any step card to see the technical specifications and a comparison with standard platform capabilities.

Step 1 twitchio

Chat Message Ingestion

Captures live chat events instantly. Bypasses standard LLM processing for pre-configured bypass lists.

🛡️ Trusted Bypass Gate
Step 2 Redis

Short-Term Context

Pushes messages to a rolling 50-message buffer in Redis to supply conversation context to the LLM.

Step 3 pgvector

Semantic Rule Match

Converts rules to vector embeddings and queries PostgreSQL using HNSW similarity lookup.

Step 4 Ornith 9B

LLM Decision Brain

Evaluates context and rule vectors. Whitelists actions before generating execution commands.

🛡️ Action Whitelist Shield
Step 5 MCP Server

Silent Execution

Spawns the MCP subprocess to call the Helix API silently in a local, network-isolated workspace.

🛡️ Stdio Sandbox
Step 6 PostgreSQL

Encrypted Audits & Logs

Logs decision hashes, action details, and incoming client IPs securely in PostgreSQL.

🛡️ Encrypted Audit Logs
Step 1 OBS Web App

NL Streamer Request

Streamer enters a command on the OBS dashboard. Scans for prompt injection and applies strict length filters.

🛡️ Injection Scan Gate
Step 2 Direct Ollama

Ollama JSON Output

Bypasses LangChain; sends command to Ollama for structured action JSON.

Step 3 Redis / API

ID & Message Resolution

Resolves Twitch username to ID, and looks up the target message ID in the Redis rolling buffer.

Step 4 PostgreSQL

Pending Actions Queue

Saves proposed actions to an isolated pending database table. Prevents autonomous command execution.

🛡️ Approval Gate
Step 5 OBS Web App

Streamer Review

Broadcaster reviews and clicks Approve/Deny. Dashboard validates origin and requires secure API key tokens.

🛡️ Token Authorization
Step 6 MCP Server

Action Execution

Upon approval, the MCP client spawns the tool subprocess silently with local stdio piping.

🛡️ Stdio Sandbox
Step 1 Web Audio API

OBS Mic Input Capture

Microphone capture activates on threshold breach. Deduplicates audio hashes and limits command rates.

🛡️ Voice Replay Guard
Step 2 FastAPI / Webhook

Secure Audio Routing

POSTs audio internally. Routes to webhook with automated local fallback if the cloud network is offline.

🛡️ Local Fallback Gate
Step 3 Whisper.cpp

Whisper Transcription

Transcribes audio inside the local Docker network via natively compiled whisper-server on port 8080 in ~1s.

Step 4 PostgreSQL

Fuzzy Trigger Match

Queries channel-specific triggers (e.g. "clip that") from Postgres and performs substring matching.

Step 5 Speech Extraction

Dynamic Clip Titling

Extracts trailing speech following the trigger word to set as the custom Twitch clip title automatically.

Step 6 Google Drive API

Helix & Drive Backup

Generates clip and uploads backup. Reads Google API key from mounted Docker secrets instead of env vars.

🛡️ Secret Vault Gate