This is the next how-to guide in the series. The earlier drift article explained what behavioral drift is and why it matters. This one is practical: how to set your agent up so drift is measurable, how to read the dashboard, how to tell a real change from noise, and what to do when the score moves.
It is written for engineers and team leads who ship prompt, model, or tool changes regularly and want to know, within days, whether the agent now behaves differently.
A note on scope: the SDK documentation lists baseline(agent) as the method that returns a behavioral drift signal, and the dashboard has an Agent Behavior view. I could not find a public reference for the exact scoring formula, alert thresholds, or the full return fields of baseline(), so this guide sticks to what the product shows and tells you where to verify. Check the repository docs for those details before you build automation on them.
Before you start: what drift actually measures
Drift compares an agent’s recent behavior with its own established baseline. In the dashboard, the comparison covers the shape of what the agent does: how often each event type occurs, how often one event follows another (the transitions), how long sessions run, and how often errors occur. In a typical reading, the window might be the newest 50 sessions compared against the 890 sessions before them.
Three things follow from that design:
Drift is relative to your own history. There is no universal good score, only change against your baseline.
It measures change, not quality. An agent that got faster and more accurate can still show drift.
It depends on stable data. If your event names or logging change, the comparison breaks, and the score will reflect your instrumentation, not your agent.
Step 1: Make your data drift-ready
Do this before you trust any score.
Keep event types fixed. Use a small, stable set of types such as user_message, decision, llm_start, llm_end, tool_start, tool_end, assistant_response, and error. Put variable details in the payload, not in new event names. Inventing new types per release fragments the baseline.
Name your tools. The SDK provides log_tool and log_tool_result, so the dashboard can show which tool changed instead of only that tool calls changed. The LangChain, LiveKit, and MCP integrations already do this.
python
call = await db.log_tool("support-bot", "lookup_order", args={"order_id": "ORD-8842"})
await db.log_tool_result("support-bot", "lookup_order", result=order, parent_id=call.event_id)
Link events with parent_id. Transitions come from parent and child relationships, so a missing parent link silently distorts the transition view.
Use a stable agent name. Drift is computed per agent, so renaming an agent starts a new history.
Step 2: Let a baseline form
Drift needs history to compare against. Run the agent in real conditions long enough to collect a meaningful volume of sessions before you rely on it. There is no magic number, but a baseline built from a few dozen sessions is fragile, and one built from several hundred is far steadier. If your traffic is low, widen the window rather than reading tiny samples.
Be careful about what the baseline contains. If it spans an earlier version with very different behavior, today’s drift score partly reflects that old behavior. After a deliberate, large change, expect the score to stay elevated until enough new sessions accumulate.
Step 3: Open the Agent Behavior view and choose a window
The dashboard offers windows for Sessions, 24h, 7d, and 30d. Pick based on your traffic:
Sessions compares the most recent batch of sessions against older ones. Good for steady traffic and for checking a release.
24h and 7d suit agents with daily patterns, such as business hours, where comparing by session count could mix very different times of day.
30d shows slow movement that shorter windows can miss.
Use the Recompute button after a window change or after new data arrives. If the window is small, treat the result as a hint, not a verdict.
Step 4: Read the headline, then the health context
The top of the view gives a drift label, a percentage, and a score. In one reading it looked like this: minor drift, 9 percent behavior change, score 0.090, with the note that small shifts in event distribution are worth a quick look.
Read that as triage, not diagnosis. Then read the health context next to it, because it tells you whether the change did damage:
Sessions analyzed, which shows how many recent sessions were compared with how many prior ones. Check this first. A tiny recent window makes everything noisier.
Error rate delta, shown in percentage points. A negative number means fewer errors.
Average session length, shown as a percentage change, with a note on whether it is within a normal range.
A drift score that is up while errors are down and sessions are shorter points toward a change in approach, not a failure. A drift score that is up while errors are up points toward a regression. The combination matters more than any single number.
Step 5: Read the biggest changes
This is where the diagnostic value sits. The view ranks the events and transitions that moved the most, in percentage points, between the recent window and the baseline. Each row shows whether it is an event or a transition, the before and after share, and the change.
In the earlier example, the largest movers were:
The share of assistant_response events fell from 11.51 percent to 6.78 percent.
The transition from decision to assistant_response fell from 4.13 percent to 0.00 percent.
tool_start, tool_end, and the transitions between llm_end, tool_start, and tool_end all rose by roughly 2 percentage points.
Turn that into a plain sentence before doing anything else: the agent now goes through tools more often and goes from decision to response directly far less often. That sentence is your hypothesis.
Two cautions when reading the list:
A transition dropping to exactly zero deserves suspicion. Behavior rarely changes to precisely nothing. It may mean the code path was removed on purpose, or that something stopped being logged, such as a changed event name or a missing parent link. Check your instrumentation before you assume the agent changed.
Look at shares, not only counts. Shares are percentages of all events, so when one event type grows, others shrink, even if nothing happened to them. Read related rows together.
Step 6: Compare the baseline and recent panels
Below the ranking, side-by-side panels show each period in full: sessions and events analyzed, events per session, average duration, error rate, the event distribution, and the most common decision sequences, meaning parent to child pairs.
Use them to check your hypothesis against raw numbers. In the example, events per session went from 12.3 to 10.6, average duration from 56 seconds to 51 seconds, and error rate from 1.2 percent to 0.6 percent. Fewer events, faster sessions, and fewer errors fit a story of a more direct, more efficient path, which makes the shift look benign, though not yet understood.
Step 7: Pull the signal programmatically
For automation, the SDK exposes the drift signal as baseline(agent):
python
from zizkadb import ZizkaDB
async with ZizkaDB(host="http://localhost:8000") as db:
result = await db.baseline("support-bot")
print(result)
Print the full result first and read its fields before you write any logic against it, since I could not confirm the exact structure. Once you know it, you can run the check in a scheduled job, a release pipeline, or a notebook, and post a summary to your team channel.
Step 8: Decide whether the change was expected
Drift only means something against what you intended. Before investigating, answer one question: did we change something deliberately in this period?
Keep a simple change log with date, change, and owner, covering prompts, models, tools, retrieval settings, routing logic, and upstream provider updates you know about. Then sort each drift finding into one of three groups:
Expected: you changed something, and the shift matches what you meant to change. Confirm it and note it.
Unexpected but explained: you did not intend it, but you can find a cause, such as a provider model update or a changed tool response.
Unexplained: nothing in your change log fits. These are the ones to investigate first.
Step 9: Investigate with replay, lineage, and point-in-time state
When a shift needs explaining, use the other functions in order:
Use Session Replay on a few sessions from the changed period, and compare them with sessions from the baseline. Look for the specific path that changed, such as a verification step missing or an extra tool call.
Use zizkadb why or db.why() on the step where the new path begins, to see what inputs and context led there.
Use db.at() to reconstruct what the agent looked like around the time the shift began, including prompt, tools, and model, and compare with an earlier time.
Check the tool and API error rates for the tools involved, since a tool that started failing often produces a shift to different paths.
Write down the cause category: prompt change, model change, tool behavior, retrieval change, user mix change, or instrumentation change.
Step 10: Decide, act, and record
Each investigation ends in a decision:
Accept. The change is intended or harmless. Note it, and consider whether the baseline should now be understood as starting from this change.
Fix. The change is harmful. Adjust the prompt or tool, add a check, or roll back, then watch the score and the biggest-changes list for the following days to confirm it reverses.
Watch. You cannot tell yet. Set a review date, and widen the window if volume is low.
Record the outcome in the same change log. Over time this becomes the history that tells you which kinds of changes move your agent most.
Step 11: Turn it into a routine and a release check
A single look is not a process. Set a light routine:
Review the drift view after every prompt, model, or tool change, and once a week otherwise.
Pick a few transitions that must never disappear, such as a verification step before a payment, and check them by name each time.
For releases, run your usual scenarios and compare behavior against the baseline before and after. LLM output is not deterministic, so use enough runs to separate noise from signal, and treat the score as a trigger for review, not an automatic pass or fail.
Tighten human review temporarily when drift appears, for example by sampling more of the agent’s actions until you understand the change.
If you run many agents, ZizkaDB’s Fleet feature is described as covering multi-agent monitoring across projects. Check the current docs for how it presents drift across agents.
What the business case looks like
These figures are modeled assumptions, not measured results. Replace them with your own.
Assume a team ships 3 significant prompt or model changes a month at 100 euro per engineering hour. Without drift monitoring, a harmful change is typically found through ticket volume after about 10 days, and cleanup takes around 40 hours, costing 4,000 euro per incident. With drift flagged within about 2 days, the fix is more targeted and takes about 6 hours, costing 600 euro. If one change in three turns out harmful, the monthly saving is roughly 3,400 euro.
Add investigation time. If a team handles 5 behavior investigations a month, starting from a ranked list of changed events instead of a blank page might reduce each from 4 hours to 1, saving about 1,500 euro a month.
The harder number is the incident that never reaches a customer. A regression caught in two days rarely produces complaints, refunds, or churn. That is difficult to model honestly, so measure it on your own releases before counting it.
Common mistakes to avoid
Treating the score as a verdict. It says behavior changed, not whether that is good.
Reading small samples. A window of a handful of sessions produces noise. Check the sessions analyzed count first.
Changing event names between releases. The baseline fragments, and the score reflects your logging.
Ignoring a transition that drops to zero. Check for a removed code path or a logging gap before assuming behavior changed.
Skipping the change log. Without it, you cannot sort expected shifts from unexplained ones.
Forgetting the baseline contains history. After a deliberate large change, the score stays elevated until new sessions accumulate.
Using drift alone. Pair it with error rate, session length, and replay of real sessions.
Where to go from here
Open the Agent Behavior view for your busiest agent today. Read the sessions analyzed count, the health metrics, and the top three changes, then write your hypothesis in one sentence and check it with a replay. After that, start a change log and add a weekly review. The earlier guides on incident investigation, point-in-time reconstruction, and oversight all connect to this, since drift is usually the first signal that sends you to them.
If you tell me how often you ship changes and what the agent does, I can help you choose which transitions to guard and how to set up the review routine.
Want to test ZizkaDB on your agent? Try our open source version here: https://github.com/ZIZKA-AI-SL/ZizkaDB
Interested in a design partnership? Fill in the form on our site or reach me directly at founder@zizka.ai
Sources: ZizkaDB Python SDK wiki · ZizkaDB repository README
