Setting cut scores and interpreting candidate results
Three ways to establish your cut score, what your candidate volume lets you say with confidence, and what happens once you're live.
A cut score is the score threshold that separates candidates who advance from those who don't. If the cut is 60 out of 100, a 62 moves to the next stage of the hiring process and a 55 doesn't. In pre-hire assessment, it's the bar a candidate must clear to be considered further.
Set well, a cut score makes the screen fair, objective, and traceable to the role's actual requirements: every candidate is measured against the same bar, and the decision comes out the same no matter who reviews it. Setting it well is the work. This guide covers how we do it together.
1 · Deriving your cut score
Three ways to establish your cut score
Method 1
The consult
Minimum viable candidate
A 15 to 30 minute conversation with your engineering leader or hiring manager: what can a minimum viable candidate do? Not your best engineer, the weakest one you'd still hire. We translate that against the rubric into a minimum score: your competence floor.2 Below it, no candidate advances, regardless of how the pool looks.
Method 2 (optional)
Engineer calibration
Internal team benchmark
1 to 3 of your engineers complete the assessment, validating the floor against real internal performance instead of judgment alone.2
Method 3
The funnel math
Future-stage interview capacity
Interview capacity ÷ expected applicants = the pass rate the assessment must produce, which implies a percentile cut.
Together: your cut is whichever is higher, the competence floor or the funnel's percentile. Each method alone gives you a defensible starting point, and the more of them you stack, the stronger your team's understanding of what makes a passable candidate.
The end-to-end flow
1
Consult 15 to 30 min
Your engineering leader or hiring manager defines the minimum viable candidate. We translate it into a competence floor via the rubric, and collect the funnel numbers while we're there.
Topic
What we ask
The work itself
What does this engineer actually do in a typical week? Which tasks, done badly, hurt the team most?
Skills & knowledge
Which technical skills are non-negotiable on day one, and which can be learned on the job?
The minimum bar
What can a minimum viable hire do? Where have past mis-hires fallen short?
Assessment fit
Which parts of the assessment map to the work above? Anything critical it doesn't touch?
Funnel context
Expected applicant volume for this req, and how many interviews the panel can actually run?
2
Engineer calibration Optional · ~1 hr per engineer
1 to 3 of your engineers complete the assessment, upgrading the floor from judgment to real internal benchmark data.
3
Derivation session 45 min
Funnel math done live with your recruiting lead. The cut lands as the higher of the floor and the funnel's percentile, and everything goes into a one-page memo the same day.
Worked example
200 applicants, 12 onsites → 6% pass rate → cut near the 94th percentile.
Same panel, 40 applicants for a niche role → 30% pass rate → 70th percentile.
4
Go live
Candidates flow. Clear scores advance or don't; scores near the line route to your review. Consult through go-live typically fits inside the first two weeks of a pilot.
5
Recalibrate on evidence After first candidates, then quarterly
Once real candidates have moved through interviews, we compare scores against actual outcomes and adjust. Cuts start at the lower end of the defensible range and move up as the data supports it, which also minimizes adverse-impact risk.3
2 · What your volume unlocks
Your read on the candidate pool sharpens as sittings accumulate
0 to 29 sittings
Rubric-anchored read
71/100n=12
Score read against the role's rubric and your floor
Do
Read raw scores against the rubric and your competence floor. Review everyone near the line.
Don't
Show percentiles, make pool-level assumptions, or auto-reject at a cut.
e.g. Floor set at 60 in the consult. A 72 advances, a 45 doesn't, a 58 gets a human look at the work.
30 to 99 sittings
Quartile rankings unlock
71/100Top quartilen=74
vs. all candidates on this question
Do
Screen on quartile bands (top quartile, top half) with a review band at the cut.
Don't
Trust single-point percentiles or hard auto-reject.
e.g. "Top quartile" means schedule first. Don't debate a 62 vs. a 58: at this volume the gap is within measurement noise.
100+ sittings
Pool distribution unlocks
62/10062nd percentile (52 to 72)n=183
Do
Use numeric percentiles. Your derived cut score now works with precision, with a narrow review band.
Don't
Slice norms by sub-group: each slice resets the count.
e.g. Cut at the 65th percentile, review band roughly 58 to 72. Above 72 clears the bar, below 58 doesn't, in between gets reviewed.
Why the bands: same score, different confidence
A candidate scores at the 62nd percentile. The shaded band shows where their true standing could fall (95% confidence).1 More sittings, tighter band, stronger decisions.
<30 sittings
±18 pts
44th to 80th
~100
±10 pts
52nd to 72nd
~200
±7 pts
55th to 69th
~500
±4 pts
58th to 66th
Ranges use the standard statistical method for percentile estimates (binomial confidence intervals), computed at the middle of the distribution where uncertainty is widest.1 Sittings pool across all companies using the same question version, so your effective sample is usually larger than your own pipeline. Only the anonymous score distribution pools; each candidate's own work stays with the hiring team that invited them.
3 · Once you're live
Humans decide at the edge
Candidates clearly above the cut advance; clearly below, they don't. Scores within measurement noise of the line route to your review: whoever you choose, typically the hiring manager or a senior engineer, looking at the actual work rather than just the number.
The band is wide when data is thin and narrows automatically as volume grows, so the review load shrinks over time. Candidates far from the line are correctly classified even at low volumes; the risk lives at the boundary, and the review band absorbs it.
Below
Review
Advance
Does not advanceHuman decisionAdvances
4 · Calibrating over time
The cut score improves with your hiring data
Going live starts the calibration loop; it doesn't end it. Once enough candidates have moved through the full hiring process, we look at how assessment performance lined up with what happened next: interview ratings, performance in later rounds, whether offers were made, group differences,3 and eventually on-the-job performance. Those comparisons show whether the cut is sitting where it should.
If the cut needs to move, the change waits until the roles hiring under the current cut have closed, so every candidate in a given req is measured against the same bar. Adjustments start from evidence, land between reqs, and are documented in the same memo that set the original cut.
5 · FAQ
Common questions
Why don't I see percentiles yet?
Your question hasn't crossed 30 pooled sittings. A percentile at that volume carries ±18 points of noise: it would look precise while being unreliable. Quartiles unlock at 30, numeric percentiles at 100.
Where does the cut come from before we have volume?
From the consult: your engineering lead defines what a minimum viable candidate can do, translated against the rubric. Engineer calibration can validate that floor with real internal scores. Either way, it re-anchors to candidate data as sittings accumulate.
Do we generate all this volume ourselves?
No. Sittings pool across every company using the same question version, so norms mature faster than any single pipeline. Only the anonymous score distribution pools; your candidates' own work stays with you.
Why did two reqs get different cuts?
Different funnels. The percentile cut is derived from applicant volume against interview capacity, so a high-volume req cuts higher than a niche one, deliberately.
Is this legally defensible?
The cut is set by recognized standard-setting methods,2 documented with its derivation, set at the minimum defensible level, and monitored for adverse impact at recalibration.3
What does the reviewer actually review?
The work product: the design, the debug, the analysis, not the score. Simulation assessments leave an artifact worth inspecting; that's the point of the review band.
A candidate scored 62. Is that good?
Depends on volume. At 100 sittings it means somewhere between the 52nd and 72nd percentile. The report always shows the range, never just the point.
Method notes & references
Percentile confidence intervals use the binomial method (Woodruff, 1952; Korn & Graubard, 1998), the approach used in federal statistical reporting. Intervals shown are at the median, where uncertainty is widest; extreme percentiles are tighter.
The consult follows modified-Angoff practice (expert judgment of minimally acceptable performance); engineer calibration follows the Contrasting Groups method (incumbent data). Both are recognized standard-setting methods (Livingston & Zieky; Cizek & Bunch). Method choice materially moves the cut. One clinical study found cuts from 66% to 86% on the same exam depending on method, which is why the derivation is documented, not just the number.
Consistent with EEOC Uniform Guidelines practice: monitor selection-rate ratios (four-fifths rule) and set cuts at the minimum level justified by business necessity.
