A proof of concept that scores 98% on a folder of images tells you almost nothing about whether the system will work on the line. Nearly every stalled vision project we have been asked to rescue failed for a reason that had nothing to do with the model.
1. Lighting nobody controlled
Ambient light changes across the day, across seasons, and when someone opens a bay door. If the proof of concept was shot under one condition and the line runs under several, accuracy will drift and nobody will be able to say why. Fix: controlled, enclosed lighting specified during the survey, and capture across every condition the line actually sees.
2. A camera mounted where there was room
Mounting position gets decided by what is convenient rather than what the optics need. Angle, working distance and vibration all move the result. Fix: treat mounting as an engineering deliverable with a drawing, not a task for install day.
3. Classes nobody agreed on
Two inspectors will disagree about borderline defects, and if they do, the labels will contradict each other and cap the achievable accuracy. Fix: write the class definitions with the quality team before annotation begins, and label a sample twice to measure how often they agree.
4. Changeovers and new products
The system is validated on the current product mix, then a new part number arrives and accuracy quietly falls. Fix: plan the retraining path in the original scope and monitor for drift rather than waiting for the quality report.
5. Results that reach nothing
A detection that appears on a dashboard changes no outcome. It has to reach the reject mechanism, the work order queue, or the supervisor. Fix: specify the integration and the fail-safe behaviour before the model work starts.
6. No owner after go-live
The project team disperses and the system belongs to whoever sits nearest. Fix: name the owner during the project, and put monitoring and a retraining schedule behind them.
The pattern
Five of these six are decided before any model is trained. That is why our engagements start with a site survey rather than a dataset.