Listen
A participant says what they think the question means and how they chose an answer.
Questions being tested
Before an assessment question is used, people need to understand it as intended.
This page shows the draft questions being tested, the standards they have not yet met, and the review required before they can change a scored pilot.
Now testing
Two unscored drafts.
Ownership Index
0.3.0-candidate.1
Portfolio Professional
0.2.0-candidate.1
The rule
The interview tests the question. It does not test, score, diagnose, or rank the person answering it.
From draft to decision
Find friction
We look for confusing words, missing choices, inaccessible formats, and situations the question fails to fit.
Rewrite
A revised question goes to new participants. The same person is not asked to approve our fix.
Test
Only then do we examine response quality, reliability, structure, and fairness across relevant groups.
Publish the decision
An adopted set receives a new version number and an explanation. A failed question is revised or rejected.
Why a question is tested before it is scored
A question can be answered consistently by everyone and still measure the wrong thing. If half of the people answering read “audience” as everyone who follows an account and the other half read it as everyone who reads the work, both groups will answer confidently and the resulting number will mean two different things. That failure is invisible in the response data, because nothing about it looks like an error. It is only visible when someone is asked to say, in their own words, what they thought the question was asking and how they arrived at their answer.
What counts as friction
A draft is held back when a word turns out to carry more than one common reading, when the answer choices leave a real situation with nowhere to go, when a question assumes circumstances not everyone has — a team, a budget, a stable address, a single employer — or when the format itself is the obstacle rather than the content. Each of those produces an answer, which is why response rates alone never surface them.
What a version number means here
A candidate version is a draft under test and is never scored, shown as a result, or compared against a published version. A question only moves into a numbered pilot once it has survived rewriting and a fresh round of participants, and the change is published with the reasoning attached. This is also why a pilot version can go up without any new questions appearing: removing a question that did not survive is as much a version change as adding one.
Help us find unclear or unfair questions
Tell us where a question is hard to answer.
Tell us when a word is vague, none of the choices fits your situation, or a question assumes resources you do not have.
See the interview study