Moderation
What the platform checks before strangers read it, how to switch it on, and how moderators act on what players report. The code is apps/api/src/moderation.ts (the moderators), apps/api/src/takedowns.ts (the moderators' routes) and apps/nakama/src/index.ts (the chat hook); the routes are listed in Platform.
What is checked
| Text | When | Refused |
|---|---|---|
A game's texts: titles, blurbs, items, objectives, messages, dialogue lines, the phone's apps (textsOf) | when its draft is published publicly (POST /drafts/:id/publish) | 422 with the reasons; the MCP's draft_publish says them, and nothing is published |
A generated level's district titles, which the minimap shows (levelSpecTexts) | the same | the same |
| A line of a public room's chat | as it is said: the game server asks POST /moderation/chat with the room secret (under openrouter, the word list alone) | the line is hidden; a moderation that cannot answer hides it too |
A message a client writes straight to Nakama on a room's channel (room-…) | Nakama's runtime module asks the same route before delivering it | dropped |
| A creation's brief | when it is queued (POST /jobs), before its creation is spent and any model runs | 422 with the reasons (reason: "moderation"); nothing spent, nothing queued |
| What the platform's model wrote: its draft's texts, its level spec's, its last words | when the creation ends (finished, cancelled, its runner gone), before its owner sees any of it | the creation fails with why; nothing of it is shown, its creation stays spent (below) |
A game the platform's model wrote (byModel) | when it is published, by link too | 422 with the reasons, as a public game |
Not checked: other games shared by link and their rooms (playtests too: only who has the link sees them), players' display names, the texts of reports, and the party's and direct messages (private between friends). Players report what slips through, and moderators decide (below).
The modes (FOI_MODERATION)
| Mode | What it does | Needs |
|---|---|---|
off | Allows everything. The default, for local development; the API warns at start. | nothing |
wordlist | Refuses a text with a word from the list (apps/api/blocked-words.json, plus FOI_BLOCKED_WORDS, comma-separated). Instant, offline. | nothing |
claude | Asks a small Claude model to classify the texts against the game's tone guide (the "never" list of GAME_DESIGN §15). A failure to reach the model refuses, so nothing slips through. | ANTHROPIC_API_KEY |
chain | The word list first, Claude on what passes: what production should run. | ANTHROPIC_API_KEY |
openrouter | The word list first, then a small model on OpenRouter with the platform's own key (no provider that keeps the texts for training); what each call cost goes into the ledger of the platform model's spend, so it counts in the creations' month (FOI_CREATOR_MONTHLY_USD). A failure to reach it refuses. It checks the briefs, what the model writes and the games it made when they are published, and public games; public rooms' chat is checked by the word list alone, so its lines never spend the creations' budget. | OPENROUTER_API_KEY |
FOI_MODERATION_MODEL picks the classifier (claude-haiku-4-5 for claude and chain, google/gemini-2.5-flash-lite for openrouter, by default). The API refuses to start with claude or chain and no Anthropic key, or openrouter and no OpenRouter key, and the Compose stack's preflight says so before anything starts. With openrouter the chat of public rooms is checked by the word list alone (createChatModerator; /health says chatModeration: "wordlist"): the classifier is a paid call counted in the creations' month, kept for the briefs, what the model writes and the games published, never spent on the chat's volume.
Creations
When the platform's own model creates games (FOI_CREATOR, Platform), the moderation checks it at both ends, and creation on OpenRouter or Claude does not start without one (wordlist at least; the stack's preflight refuses it):
- The brief, before any model runs. After the budget and before the creation is spent: a refused brief is turned away with the reasons and costs nothing.
- What the model wrote, before its owner sees it. While a creation runs, its draft is shown to nobody but the creation's own session. When it ends, its texts, its level spec's and the model's last words are checked: passed, the result is shown; refused, the creation fails with why, its creation is not given back, and nothing of it is shown: a draft it made is withheld (404 to its owner, never listed), a draft it revised is put back as it was before (a revision on), and its builds are withheld (their artifacts carry the texts). A creation cancelled while it runs is checked the same way before its draft so far is shown.
- When it is shared. A game the platform's model wrote is checked when it is published by link as well as publicly, so its owner's own edits pass the same test before anyone else sees it.
Transcripts. Each creation's conversation (the brief, the model's words and tool calls, the tools' answers) is kept 30 days for the moderators and the admins, then forgotten: GET /api/moderation/creations/<job>/transcript (signed in as one) answers the creation (its owner, its brief, how it ended, whether it was withheld, what it cost) and its transcript. From a report: GET /api/moderation/creations?game=<g-…> lists the reported game's creations (its draft's), with their ids; the admins' page lists every creation too.
Switching it on in the Compose stack
In deploy/.env:
FOI_MODERATION=chain
ANTHROPIC_API_KEY=sk-ant-… # a key of your own Anthropic account
FOI_MODERATION_MODEL=claude-haiku-4-5
FOI_BLOCKED_WORDS= # more words for the list, comma-separated
FOI_MODERATORS= # the moderators' user ids (below)Then npm run stack:up (or docker compose up -d --wait from a release's bundle). Check it took:
curl -s https://<domain>/api/health # "moderation": "chain"On a stack with real secrets (FOI_ALLOW_DEV_SECRETS=0), the preflight warns loudly while moderation is off: a public game's texts would reach strangers unchecked.
Moderators
Moderators are accounts whose ids FOI_MODERATORS lists, comma-separated: the Nakama user id on the stack, user-<email> with development accounts. An account learns its id from GET /api/me, which also says "moderator": true once it is one:
curl -s https://<domain>/api/me -H "Authorization: Bearer $FOI_TOKEN"Invoke-RestMethod -Uri "https://<domain>/api/me" -Headers @{ Authorization = "Bearer $FOI_TOKEN" }Set the id in deploy/.env and restart the API (npm run stack:up).
Moderators also read the crash reports the players' games send with their consent, grouped by fingerprint, the most frequent first (days from 1 to 90, 7 by default):
curl -s "https://<domain>/api/telemetry/crashes?days=7" -H "Authorization: Bearer $FOI_TOKEN"Invoke-RestMethod -Uri "https://<domain>/api/telemetry/crashes?days=7" -Headers @{ Authorization = "Bearer $FOI_TOKEN" }There is no moderators' page yet: a moderator calls the routes with their session ($FOI_TOKEN, from POST /api/auth/login), or asks Claude to. The open reports, then a takedown with a note:
curl -s "https://<domain>/api/moderation/reports?state=open" -H "Authorization: Bearer $FOI_TOKEN"
curl -s -X POST "https://<domain>/api/moderation/games/<gameId>/takedown" -H "Authorization: Bearer $FOI_TOKEN" \
-H 'content-type: application/json' -d '{"note": "real people named"}'$auth = @{ Authorization = "Bearer $FOI_TOKEN" }
Invoke-RestMethod -Uri "https://<domain>/api/moderation/reports?state=open" -Headers $auth
Invoke-RestMethod -Method Post -Uri "https://<domain>/api/moderation/games/<gameId>/takedown" -Headers $auth `
-ContentType "application/json" -Body (@{ note = "real people named" } | ConvertTo-Json)The other routes (restore, featured, dismiss, the audit log) take the same shape: Platform. A takedown keeps a game's page, versions and preview from everyone but its creator and the moderators, opens no new room of it and lets no one join its open rooms; every decision goes into the audit log (GET /api/moderation/audit), which nobody edits.