By Tim O.
I am studying for the AWS Certified AI Practitioner exam, and I wanted something to build rather than another set of flashcards.
The 2026 World Cup made the choice easy. I am a big soccer fan, and this tournament was the first with the new 48-team format: more teams, more games, twelve groups instead of eight, and a round of 32 that did not exist before. The stats were everywhere. It also happened well after any foundation model's training cutoff, which makes it a genuine retrieval problem rather than a fake one. The model cannot answer these questions from memory. It has to look them up.
So I built a bot that only answers questions about the 2026 World Cup. Amazon Bedrock Knowledge Base for retrieval, 177 documents in S3 covering every match and every squad, a system prompt telling the model to stay on topic, and a Bedrock Guardrail as the enforcement layer.
The scoping mostly worked. What I did not expect was how it failed.
The setup
Six denied topics: other sports, betting, professional advice, cooking, software development, and writing assistance. Plus contextual grounding, which scores whether an answer is supported by the retrieved passages and blocks it if not.
Denied topics are a blocklist. You describe a subject, and the classifier decides whether the input matches. My first attempt tried to describe the boundary instead of the subject:
General assistant requests unrelated to the 2026 World Cup, such as writing code, composing emails, or giving recipes.
That topic never fired once. Asked for a carbonara recipe, the guardrail returned action: NONE.
The reason is obvious in hindsight. The classifier has no idea what my application is about. "Unrelated to the 2026 World Cup" is meaningless to it. Rewriting the same topic as three concrete subjects, named the way an outsider would name them, fixed it immediately.
Then it blocked my own use case
I had a denied topic for club football, to catch questions about the Champions League and transfers. It also blocked this:
What was the final table in Group F?
That is not an edge case. That is the single most common question my bot exists to answer.
So I ran the same sentence across all twelve groups.
Ten allowed, two blocked. Same sentence, same guardrail, one letter different.
I reworded the offending topic. The blocks moved to different letters. I removed that topic entirely, and a different topic, one about other sports, started blocking a group table question about football. I reworded that one too. The blocks moved again.
Running the sweep on two different days gave two different sets of blocked letters.
The false positive rate stayed roughly the same throughout. Rewording relocated the errors without reducing them.
The grounding filter has the same problem, in reverse
Contextual grounding scores your answer against the retrieved passages. Set the threshold high and correct answers get blocked. Set it low and fabrications get through.
Here is a source document, and the answer the guardrail refused to return:
The passage was retrieved correctly and the figures in it are right. The answer was blocked anyway.
Lowering the threshold let tables through, and that is when the other failure showed up. Asked for the same group table twice at temperature zero, the model produced two different sets of wrong numbers. The source document was retrieved correctly both times. Only the top row survived intact.
Grounding checks whether your answer traces back to the source. It does not check whether you copied the numbers correctly.
The hallucination it could never catch
This one is my favorite.
The source lists three goalkeepers in order. Santiago Mele is the second. The bot called him the third-choice goalkeeper and cited the passage.
There is no depth chart anywhere in my data. The squad list gives names and positions, nothing more. The model did not invent a person. It invented a relationship between real people, and every token in that sentence traces back to the source, so the grounding check had nothing to object to.
Adding an explicit rule to the system prompt did not help. Neither did lowering the temperature to zero.
What actually holds the line
Three mechanisms, three different outcomes:
The guardrail caught the recipe. The system prompt and the empty retrieval caught the corner count. Neither one alone would have caught both.
What I would tell someone starting this
A denied topic cannot express "only answer about X." It is a blocklist of subjects, so define each one the way a stranger would describe it, never in terms of your own application.
Topics adjacent to your domain are the ones you can least afford. A classifier separating football at clubs from football at a World Cup is working with a distinction too fine for the vocabulary involved. Put those in the system prompt and save the guardrail for subjects that are genuinely far away.
Test with in-scope questions. Everyone tests the off-topic ones. I found every problem in this post by asking the bot things it was built to answer.
And retrieval grounding is not accuracy. Right document, right passage, wrong number in the output is a completely reachable state.
One caveat worth stating. All of this ran on Amazon Nova Lite, because Claude access on my account was still pending. The guardrail behaviour is model independent, since denied topics evaluate the input before generation. The table fabrication and the ignored prompt rules may not be. Rerunning the same tests against Claude Sonnet is the next post.
The bot works. It answers World Cup questions correctly and refuses almost everything else. But the parts I thought were the safety net turned out to need a safety net of their own.
Code and full findings: github.com/ptokito/wc2026-bedrock-bot
Part of a series building a scoped GenAI assistant on Amazon Bedrock while preparing for the AWS Certified AI Practitioner exam.
