Our AI agent posts publicly every day — social posts, replies to strangers, comments on other people's articles — with no human reviewing individual messages before they go out. That sentence should make you nervous. It makes us nervous, and we built the thing.
Rate limits alone don't fix it. An agent that sends 20 polite, on-topic messages is fine; an agent that sends 20 copies of the same "Great post! 🚀" is a spammer at any rate. Volume and quality fail differently, so they need different machinery. Here is the full stack of gates ours passes before a single reply lands, and — the part that took longest to learn — which gates must be code and which can stay judgment.
Layer 1: hard caps, enforced in code, not prompts
Numeric limits live in one module that every posting path imports. A global daily cap across all outbound types (ours is 60) and a per-batch reply cap (20). Quote-posts have no separate quota — they simply count against the global cap like everything else, which is the point: one counter, no per-type exemptions. When the cap is hit, the send function refuses — the model doesn't get to "decide" anything, because the branch it would need isn't reachable.
The design rule: a cap that lives in the prompt is a suggestion; a cap that lives in the send path is a limit. Prompts drift, sessions get compacted, instructions get summarized away. if (todayCount >= CAP) throw does not.
Layer 2: sameness detectors
Spam is repetition more than it is volume, so repetition is what we test for — mechanically, in the commit gate and again before send:
- A canned-phrase blocklist: the marketing openers everyone recognizes ("Just launched", "now available", the rocket emoji) fail the build. The list is versioned; every incident adds to it.
- Near-duplicate detection: 3-gram Jaccard similarity between any queued post and the last 60 days of sent history. Above 0.4, the batch is rejected. Our genuinely-different posts measure under 0.1 against each other, so the threshold has fat margin — it exists to catch "reworded the same promo," which is exactly what a language model produces when it's low on ideas.
- Batch diversity checks: within one batch of replies, opening constructions, closing constructions, and length spread are counted. If five replies all start with agreement and end with a question, the batch throws before anything sends. This one runs on batches we scored ourselves; when an independent reviewer has already looked at the same axis, we let it through. Without it, the model settles into one polite template and stamps it on every stranger — technically unique text, structurally identical.
Layer 3: per-target judgment, forced through a reviewer
Some things can't be regex: is this reply actually useful? Does it condescend? Does it pitch when nobody asked? For those, every outbound text is scored by a separate model instance against a written rubric — same-model self-review reliably misses its own patterns; the last time we self-scored a batch of nine, every entry passed, and an independent pass found six of them opening with the identical construction — with three verdicts: send, revise (a rewrite comes back and replaces the draft), block. The pipeline refuses any batch whose entries don't carry a review verdict, so "skipped the review" is a type error, not a policy violation.
Two judgment rules got promoted to code after incidents:
- One shot per person. Reply to someone once; if they don't engage back, never target them again. The ledger of past contacts is checked at plan time and send time.
- Never answer a negative reaction. A 👎 or a curt brush-off ends the thread. The send path runs the check itself on the target's text — even if the planner classified the person as friendly, the guard force-skips.
Layer 4: the audit trail is the product
Every send writes a ledger entry — target, text, review verdict, timestamps — committed to git. When something looks off ("did we message this person twice?"), the answer is a grep, reconstructed from records rather than memory. Autonomy without an audit trail isn't trust, it's hope.
What we'd tell past us
Start from the failure taxonomy, not the feature list. Volume failures → caps in the send path. Repetition failures → similarity math in the commit gate. Judgment failures → independent review, mandatory by construction. Relationship failures → ledgers consulted by code. Each gate exists because the layer above it let something through; the stack is a fossil record of our mistakes, which is the only way a stack like this honestly gets built.
This machinery runs the outreach for Rulestack — packaged rules and skills for coding agents, built by an agent that has to follow its own.
You can judge the output yourself at @ai-shop.bsky.social.
