Sayash Kapoor and Arvind Narayanan surveyed fields that adopted machine learning and found data leakage in 17 fields, affecting 294 papers, in some cases “leading to wildly overoptimistic conclusions” (Patterns, 2023). Those were peer-reviewed researchers. A student’s project claiming 98% accuracy deserves the same scepticism, and checking it takes about fifteen minutes. The accuracy figure is the least informative line in an ML résumé. How the student split the data, what baseline they beat, and what the model gets wrong tell you much more.
Key takeaways
- Leakage is common in published work, not a beginner’s quirk. When it was corrected in one field, complex models did no better than decades-old logistic regression.
- Of 18 neural recommenders from top conferences, only 7 could be reproduced, and 6 of those were often beaten by simple heuristics. Always ask for the baseline.
- Where the data came from matters as much as the model. None of 62 screened COVID-19 imaging models was judged of potential clinical use.
- Four questions (baseline, split, preprocessing, a failure case) tell you whether a student understands their result.
Leakage is the default failure, not the exotic one
Kapoor and Narayanan’s taxonomy has eight types. Most are mistakes a student makes in a notebook without noticing: no separate test set; preprocessing on the training and test data together; feature selection on both; duplicates that appear on both sides of the split; features the model could not legitimately have at prediction time; temporal leakage, where the future trains the past; non-independence between training and test samples; and sampling bias in the test set (Kapoor and Narayanan).
Their civil war prediction case study shows what is at stake. Published work had reported that complex models beat older statistical ones. “When the errors are corrected, complex ML models do not perform substantively better than decades-old LR models.” The apparent advantage came from the leak.
For a startup, the cost of missing this is concrete. A model that leaked in the notebook will underperform the moment it meets live data, and the intern who built it will not know why.
The baseline is the question that filters
Maurizio Ferrari Dacrema, Paolo Cremonesi and Dietmar Jannach took 18 recommendation algorithms presented at top conferences. “Only 7 of them could be reproduced with reasonable effort,” and of those, “6 of them can often be outperformed with comparably simple heuristic methods, e.g., based on nearest-neighbor or graph-based techniques” (RecSys 2019). The seventh beat the baselines but not consistently a well-tuned linear method.
Google’s own guidance starts in the same place. Rule #1 of its Rules of Machine Learning is “Don’t be afraid to launch a product without machine learning,” and Rule #4 is “Keep the first model simple and get the infrastructure right.” A student who has absorbed this will tell you, unprompted, what a rules-based version or a three-line query scored. That habit is worth more than familiarity with any particular architecture.
Ask where the data came from
A Cambridge-led team reviewed machine learning models for detecting COVID-19 from chest X-rays and CT scans. Their search found 2,212 studies; 62 survived quality screening; and “none of the 62 models was of potential clinical use.” Among the problems: models “trained and tested on the same data”, public “Frankenstein datasets” merged over time so results could not be reproduced, and training sets that used images of children for the non-COVID class and adults for the COVID class (University of Cambridge). A model can score well on a set like that by learning to tell children from adults.
Student projects are built on the same kind of public datasets. The question is not whether the dataset is famous. It is whether the student knows how it was assembled and what shortcut the model might have found.
The 15-minute review
Run this on a call with the repository open. You are not grading the score. You are finding out whether the student understands it.
- “What was your baseline, and what did it score?” No baseline means the headline number has no meaning. A majority-class or simple heuristic figure is enough.
- “How did you split the data?” If the data is ordered in time and the split was random, the future has trained the past.
- “Where is the scaler fitted?” Open the code. Scaling, imputation or feature selection fitted before the split is leakage, and it is the most common kind.
- “Could the same user, patient or item be on both sides?” Duplicates and near-duplicates across the split inflate every metric.
- “Would you have this feature at prediction time?” Pick the strongest feature and ask. Features computed after the outcome are the illegitimate-feature leak.
- “Why this metric?” Accuracy on a 95/5 class split rewards predicting the majority. A good answer names the cost of each kind of error.
- “Show me one example it gets wrong.” Students who have looked at their errors can answer at once. Students reporting a number cannot.
- “Change one thing now.” Ask for a small change to the split or a feature, live. How they navigate their own code tells you whether it is theirs.
What a good answer looks like
The strongest candidate is often the one whose number went down. A student who tells you their first run looked far better than the baseline, that they then found the scaler had been fitted on the whole dataset, and that the honest margin is much smaller, has shown you the judgment you are hiring for. That is the core of our argument that self-reported proof has stopped carrying signal: a result is only worth what the person can explain about it. The same logic sits behind redesigning technical interviews for AI-assisted code. Tierones matches US startups with students from IITs, NITs and IIITs on proof of work, including project walkthroughs where the student explains the decisions in their own build, so you can start this conversation from evidence. See how it works for employers.
FAQ
What is data leakage in a machine learning project?
Leakage is when information the model should not have at prediction time reaches it during training or evaluation, so the reported score is better than the model will be in use. Kapoor and Narayanan list eight types, from having no separate test set to temporal leakage and duplicates across the split, and found it in 17 fields affecting 294 papers.
What should I ask a student about their ML project?
Ask what baseline they compared against and its score, how they split the data, whether any scaling, imputation or feature selection was fitted before the split, and which example the model gets wrong and why. Those questions take about fifteen minutes and separate understanding from a reported number.
Is a high accuracy score a good sign in a student ML project?
On its own it tells you very little. Accuracy can be inflated by leakage, can look strong on imbalanced data where predicting the majority class scores well, and means nothing without a baseline. A modest score with a clear baseline and honest failure cases is the stronger signal.
Hiring for an AI/ML role? Tell us what you need — we’ll show you proof, not CVs.