Karat surveyed 400 engineering leaders across the US, India and China and found that 71% say AI is making technical skills harder to assess — while 62% of organisations still prohibit AI use in technical interviews. Those two numbers describe the same mistake. The artifact stopped identifying its author, and most hiring processes responded by trying to police the artifact instead of changing the question. There is a cheaper move: ask the candidate to walk you through something they built, in a fixed sequence, for two and a half minutes.
Key takeaways
- 71% of engineering leaders say AI has made technical skills harder to assess; 62% of organisations respond by banning AI rather than redesigning the test.
- Structured interviews are the strongest single predictor of job performance in the revised meta-analytic estimates, at r = .42 — ahead of work samples and cognitive ability.
- Assume AI was used. Sonar’s 2026 survey of over 1,100 developers puts AI at 42% of all committed code, expected to reach 65% by 2027.
- Five thirty-second blocks — problem, constraints, approach, result, what next — plus two probing follow-ups is enough to separate a builder from a narrator.
Banning the tool is not a test
The prohibition instinct is understandable and it does not survive contact with the numbers. Sonar’s 2026 State of Code developer survey, run across more than 1,100 developers globally, reports that AI accounts for 42% of all committed code and developers expect that to rise to 65% by 2027, while 96% say they do not fully trust that AI-generated code is functionally correct and only 48% always check it before committing. A process that bans AI in the interview is selecting for a working style that no longer exists, and it still tells you nothing about the one skill the same survey implies is now scarce: knowing when the output is wrong.
The take-home has the mirror problem. Karat’s survey found take-home projects are used by 45% of US teams and only 20% in China — and whatever the regional preference, an unsupervised multi-hour brief is now the format least able to tell you who did the work. None of this means the take-home is dead. It means the take-home is no longer the assessment. It is the raw material for one.
Structure is the part that predicts
When Sackett and colleagues re-ran the personnel selection meta-analyses and corrected a long-standing overcorrection for range restriction, the ranking of methods changed. As the Society for Industrial and Organizational Psychology summarised it, structured interviews came out on top at r = .42, followed by job knowledge tests at .40, empirically keyed biodata at .38, work sample tests at .33 and cognitive ability at .31. Cognitive ability, long treated as the default, was displaced.
Read the practical implication carefully, because it is not “interviews are good.” It is that structure is doing the work: the same questions, in the same order, for every candidate, scored against the same scheme. A founder who spends forty minutes having an interesting chat about a project is running the unstructured version and getting a fraction of the signal, plus a strong feeling of certainty about it. The two-and-a-half-minute walkthrough is cheap precisely because it is rigid.
The five blocks
Give the candidate the format in advance — you are testing judgment, not ambush tolerance — and hold them to thirty seconds a block. One: the problem, stated as something a person wanted, not as a technology. Two: the constraints and what was awkward about them — the data, the API limits, the deadline, the thing that did not fit. Three: the approach and, crucially, the alternative they rejected and why. Four: the result, with whatever measurement actually exists, including “I don’t have a number for this.” Five: where it runs now, what breaks first, and what they would change.
The block that carries the most signal is the third, and it is the one candidates prepare least. Any tool can produce a working implementation; only the person who made the decision can tell you what they gave up to get it. Block five is the second-best discriminator — a project that has never met a real user has no answer to “what breaks first,” and the silence there is informative rather than disqualifying.
How to run it
- Send the five blocks with the invitation. Same brief, same order, every candidate. This is the structure that the .42 depends on.
- Let them pick the project, with one rule: it must be something they can screen-share and open. A deployed link beats a repository; a repository beats a description.
- Say explicitly that AI assistance is assumed and fine. You remove the incentive to hide it, and you get an honest account of how they actually work — which is the thing you are hiring.
- Keep the blocks to time. Interrupt at thirty seconds. Compression is the test; a candidate who cannot summarise their own work in half a minute will not write a usable standup either.
- Then ask exactly two follow-ups, both unrehearsable. “Change one requirement — say the data arrives an hour late instead of live — what breaks and what do you do?” and “point me at the ugliest part of this code and tell me why it is that way.”
- Score before you discuss. Each interviewer commits to a number per block on paper first. Anchoring is what turns a structured interview back into an unstructured one.
- Use it as a screen, not a verdict. Ten minutes end to end. It ranks a shortlist; a short paid work trial decides.
The step before the interview
A walkthrough is only as good as the pipeline feeding it. Ten minutes per candidate is cheap; ten minutes across four thousand applicants is not a process, and a founder is the hiring manager precisely because there is no recruiting layer to absorb that. That is the part Tierones handles first — verifying identity, institute and shipped work before anyone reaches your calendar, so the walkthrough is spent probing judgment rather than establishing basic facts. On why the artifacts themselves stopped carrying signal, see what happens when self-reported proof stops working, and what we check at Tierones profile.
FAQ
What is a project walkthrough interview?
A short, structured conversation in which a candidate explains something they built, in a fixed sequence: problem, constraints, approach and rejected alternative, result, and what they would change. Five blocks of roughly thirty seconds gives you two and a half minutes of core signal, which you then probe with follow-ups. It tests judgment about work they actually did rather than recall of a puzzle they may have memorised.
Does a project walkthrough work when the candidate used AI to build the project?
Yes — that is the point of the format. Assume AI was used; Sonar’s 2026 developer survey puts AI at 42% of all committed code. The walkthrough never asks who typed it. It asks why this approach and not the alternative, what the constraint was, and what breaks first. A candidate who steered the tool answers those live. One who did not, cannot, and the gap surfaces within two follow-up questions.
Is an unstructured chat about a project good enough?
No — structure is what carries the predictive power. In the revised meta-analytic estimates published by Sackett and colleagues, structured interviews were the strongest single predictor of job performance at r = .42, ahead of job knowledge tests at .40, work samples at .33 and cognitive ability at .31. The same conversation run ad hoc, with different questions per candidate and no scoring scheme, loses most of that value.
Hiring a remote intern from an IIT, NIT or IIIT and want the walkthrough to start from verified work? Tell us what you need — we’ll show you proof, not CVs.