You ask an LLM to review your setup and get back five numbered issues. Good answer. You start on #1, and a few messages later the list is a screen and a half above you.
To get to #3, you scroll up, reread it, scroll back down and type "for #3…", hoping the model knows which list you mean. Later replies bring numbered lists of their own, so "#3" can mean two different things. And nothing tracks which items you actually finished.
Why it stays broken
To the interface, a list of things to work through is just formatting. It renders and scrolls exactly like five paragraphs of explanation. Some tools now show the model's own multi-step work as a checklist, but when the list is work for the user, it's still just text.
The principle: when a reply hands the user a set of things to work through, the interface should hold onto it as work, not as text.
A fix
How you'd build it
Only the smallest version is a UI patch. Everything past it depends on the model treating the list as something it's responsible for. That makes it work for the teams that train the model and build the chat product around it, not something a front end can bolt on.
- Smallest version: a pin control on any rendered ordered list that moves it into a sticky side panel. Users check items off themselves. It's front-end only, with no model changes, and it already fixes the scrolling problem.
- Solid version: the model does the tracking. When it writes a list it marks it as actionable and gives each item a stable ID. On each later turn it reports which item it's addressing and whether it thinks that item is resolved, so the panel can highlight the active item and suggest a check for the user to confirm. Clicking an item attaches its ID to your next message. Lists with open items float on their own once they scroll out of view.
- Ambitious version: lists become part of the conversation state the model carries. Later messages can add items or split open ones, but a checked item stays checked. If new work turns up inside it, that becomes a new item marked as raised after the original closed, so nobody has to re-judge finished work. When the conversation moves on with items still open, the model asks once whether to come back to them or drop them, so skipping is a choice rather than a silent drop. Completion closes the list's span, a few messages after the last check. Spans are stored, so scrolling back through history shows each list as it stood at that point, and a thread can hold more than one.
Trade-offs to watch
- Wrong checks cost more than scrolling. Once people trust the panel they stop rereading the list, so one bad check does more damage than a missing one. Checks should start as suggestions the user confirms, and only become automatic once the model is reliably right about what's resolved.
- Not every numbered list is work. Ranked options, a recipe or a list of facts shouldn't follow you around. Auto-float should only apply to lists that ask something of the user.
- Phones don't have room for a panel. On small screens it collapses to a chip like "2 of 5 open."
- Measure what the panel hides. Track how often people reject a suggested check, and how many items are still open when a panel closes. Undo rates alone won't catch a bad check, because someone who has stopped reading the list never undoes anything.
The full post has a checklist for auditing your own product.
If you're building an AI feature that hands users steps to follow, how are you handling this?
