A difficulty label is a guess made before anyone answers. The answers tell you whether the guess was right. Difficulty shows up in two places after each question: how many people got it right, and where the wrong answers went. In the trivia quiz I build, Boardit's Trivia Tavern, the reveal after every question shows for 6 seconds how many players chose each option (checked 8 October 2026). Here is how I read that spread, and why one table's result proves much less than it seems to.
The quarter you get for free
Take a standard four-option question. Someone who knows nothing at all still picks the right answer one time in four. That is the probability of guessing a multiple choice question blind, and it means a quarter of the table answering correctly is not "hard". It is what nobody knowing looks like.
Testing has a name for stripping that luck back out: the correction for guessing. Robert Frary's 1988 guide to formula scoring covers it for exams. For a quiz table, the rough version is simple:
- share who knew ≈ (share right − 1/4) ÷ 3/4
Say three of six players get a question right. That is 50%, and the formula gives a third. So about two of the six actually knew, and the third right answer was probably luck.
So what percentage of people should get a trivia question right? There is no single answer, but a hard question should still sit clearly above a quarter. Anything near 25% tells you nobody knew it.
Six players, nobody knows: what luck does
Small tables make luck loud. Suppose six people face a question none of them knows, and all six guess. On average, 1.5 of them will be right. More usefully:
- no right answers happens about 18% of the time;
- exactly one, about 36%;
- two or more, about 47%;
- three or more, about 17%.
So nearly half the time, a question nobody knew still shows two right answers. One time in six it shows half the table right, which looks like a fair medium question. If you relabel from one night, you will move good questions for bad reasons.
Where the wrong answers went
The spread of wrong answers says more than the count of right ones. Testing calls this distractor analysis, and it is the useful half of multiple choice item analysis. These are the four shapes I look for:
- Most right, wrong answers scattered thinly. The question is easy, whatever the label says.
- The right answer and one wrong option splitting the table. This is a good hard question: the wrong option is a real belief, not a throwaway.
- One wrong option beating the right answer. Either it is a strong trap, or the key is wrong. Check the source before you blame the table; a wrong answer key looks exactly like a brilliant trap.
- Answers spread evenly over all four. Nobody knew. That is how to tell if a quiz question is too hard: not a low score, but a flat one.
The second shape is also the clearest lesson in how to write good distractors. A wrong option earns its place when people who half-know the subject choose it.
How many answers before you trust it?
A score from six people carries a wide margin. Using the Wilson score interval, which Edwin Wilson published in 1927, three right out of six gives a 95% range of roughly 19% to 81%. That range covers easy, medium and hard at once.
Pool ten tables, sixty answers, with thirty right, and the range narrows to about 38% to 62%. That is a small sample size still, but now it can tell an easy question from a hard one. Dozens of answers per question, not six, is the honest minimum.
Measuring trivia question difficulty at a live table
In my game the result stays up for 6 seconds after each question, showing the right answer and how many chose each option. Each player gets one answer per question, so every count is a person, not a second tap after a wrong first one. That makes the spread clean enough for the arithmetic above.
Six seconds is short, so jot down the counts for questions labelled hard and keep a running tally across nights. If you want to try this on real questions, Boardit, the free site where I run this online trivia quiz, is a reasonable place to start.
Relabel slowly
So how hard should trivia questions be? The labels matter less than the shapes. Easy medium hard trivia questions should each produce their own spread, and a hard one should split the table rather than flatten it.
This is a method, not a study. I have published no answer spreads from real tables, and the arithmetic above shows why one table could not settle a question's difficulty anyway. Groups differ too: a question that floors a mixed table can be easy for a table of fans.
At your next quiz with friends, write down the spread for every question labelled hard. Keep the ones that split the room, and check the key on any where a wrong option won.
Ahmed Moaz builds Boardit, a free browser game site.
