When we added DM support to our agent's feedback loop, the tempting shortcut was obvious: DMs are private, low-volume, one-to-one — surely they need less machinery than public replies. We went the other way, and the one check we added specifically for DMs turned out to be the most interesting design decision in the feature.
Same gates, shared code
Context: our agent answers feedback on Bluesky — mentions, replies, quotes, and now DMs. For public replies we had accumulated, incident by incident, a set of guardrails:
- Tone review before send — every drafted reply is scored against a rubric by an independent reviewer; unreviewed replies make the whole batch throw before any side effect.
- Batch cap — at most 20 replies per run, a shared constant.
- One-strike rule — a reaction-only or dismissive message gets no follow-up, mechanically enforced even if a reply was drafted.
-
Idempotency — a record with
respondedAtset is never replied to twice. - Daily budget — all outbound post types share one daily cap; when it's spent, replies are refused and stay unanswered rather than being marked done.
The DM implementation imports these — the same functions, not parallel reimplementations. That choice sounds like engineering hygiene, but it's really a policy statement: privacy of the channel doesn't lower the quality bar. A pushy or tone-deaf message is worse in a DM, not better, because there's no third party to see it and no community norm to absorb it.
Two public-reply mechanisms didn't transfer, and each forced a think:
Likes don't exist in DMs. Our classification for "competitor" accounts on the public timeline is "like, don't reply" — acknowledge without amplifying. DMs have no such half-gesture, so the competitor/sales class collapses to a recorded skip. The ledger entry keeps the reason (inbound-sales-dm), because a skip without a reason is indistinguishable from neglect a week later.
Thread context doesn't exist either — so we had to invent its equivalent.
The conversation-recency check
For public replies, our worst historical bug class was answering a stale snapshot: the notification hands you one post, you draft against it, and by send time the thread has moved — a third person joined, the author corrected themselves, or the author posted again. Our public pipeline re-fetches the thread at send time and drops plans that went stale.
DMs have no thread API in the same shape, but they have the same failure mode wearing different clothes: you draft a reply to message N while the sender has already sent N+1. Answering N in that state is worse than answering late — it broadcasts I'm a machine that reads one message at a time. A human never does this; they read to the bottom before typing.
So the DM responder enforces what we called the conversation-recency check: before sending a planned reply, it looks for any newer unanswered message in the same conversation. If one exists, the send is refused and the outcome recorded as stale-superseded — not failed, not done. The correct next action is to re-plan against the newest message, and the refusal is the mechanism that forces it. The plan for the old message doesn't get quietly redirected; it gets rejected, because the content of a good reply to N and a good reply to N+1 are different texts.
outcomes: replied | skipped | already-responded
| stale-superseded ← draft was for msg N, but N+1 arrived
| cap-blocked ← daily budget spent; stays unanswered
| failed
The outcome vocabulary is the design. Every terminal state a send can reach has a name, and the two easiest states to fake — "done" and "not needed" — require either a send receipt or a recorded reason.
Unanswered must stay visible
The other rule we carried over is about bookkeeping honesty: nothing gets marked responded except an actual response. When the daily budget blocks a send, the message stays unanswered in the ledger and shows up again next run. When scope or permissions block the whole channel (as they did for us for a day — the app password lacked DM access), the fetch job files one standing reminder for a human instead of failing hourly, and the count of unanswered DMs rides along in every summary notification.
It would have been easy to let blocked items drift into a soft "handled" state to keep dashboards green. But the entire value of a feedback loop is that unanswered means unanswered. The moment your ledger can say "done" without a send receipt, every number downstream of it is decoration.
Small feature, one honest invariant: a reply is either sent against the latest state of the conversation, or it is refused with a reason someone can read later. That's the whole design — for DMs, and really for everything else the agent sends.
These guardrails run inside Rulestack — an autonomous publishing pipeline that answers in public, so every reply it sends has to survive a second look.
Day-to-day notes from the same pipeline: @ai-shop.bsky.social on Bluesky.
