Manufacturing AI inspection: defect definitions, score thresholds and line interfaces

Begin inspection planning with product, defect, imaging and intended action. Offline review, operator assistance and equipment control need different data, evidence and authority. Replacing existing equipment need not be the first decision.

A hypothetical surface-inspection task illustrates the sequence, without client-line or yield results. Establish the current baseline before comparing methods.

AOI does not imply one algorithm

Automated optical inspection can use different approaches. OMRON's official AOI description includes AI capabilities, demonstrating that AOI is not inherently rule matching. It supplies no performance evidence for this project or equipment recommendation.

Evaluate methods alone or together without inferring speed, cost or capability ceilings from labels. Describe a particular rule-based device rather than every AOI system.

Assess sample coverage and reference labels

Define defects, severity, acceptable variation and disputed labels. Retain source, product, lot, equipment, lighting and annotation versions. Establish usable images and rights; resolve inspector disagreement before treating labels as references.

The official MVTec AD benchmark uses defect-free training images with normal/anomalous evaluation images. It illustrates a research option, not a required per-defect sample count or proof of production suitability.

Separate development, threshold tuning and held-out evaluation. Inspect representative products, defects and lot conditions. Augmentation or synthetic defects cannot alone establish acceptance for material real defects. Research citation is not commercial training permission; check dataset terms separately.

Define score direction before the tradeoff

Scikit-learn's threshold guide separates scoring, class decisions and actions, with task-specific metrics. Here the explicit convention is defect-positive, predicted when score >= cutoff.

For fixed examples and scores, increasing the cutoff selects no more positives: false positives cannot increase and false negatives cannot decrease. Lowering it reverses that direction. A good-part score or different comparator needs a different explanation. Precision is not guaranteed to increase monotonically.

Quality and production owners establish material-error limits and review capacity. Sector labels do not prescribe the tradeoff. A precision-recall curve alone does not establish daily workload.

Keep counts, rates and workload distinct. Average results on easy samples should not conceal rare material errors.

Measure the complete integration path

Measure capture, preprocessing, inference, communication, review and required action, including queues, concurrency and failures. Throughput does not alone prescribe an inference deadline. Evaluate lighting and fixture changes rather than assuming universal collapse.

For MES or control-system writes, confirm interfaces, part/lot association, permitted actions, acknowledgements, duplicates and safe takeover. Automatic rejection is not mandatory for every pilot; supervised offline review can be a legitimate scope.

Accept updates and use separately

Review and version operator feedback before deciding whether to train on it. Reevaluate affected samples after updates, with release, stopping and recovery owners. Do not send unreviewed feedback into live models or promise automatic yield gains. The NIST AI RMF supports lifecycle assessment, not machine-safety certification.

The existing manufacturing scope starts with actual samples, error limits and interfaces. The Copilot measurement guide separately addresses knowledge-work use, quality and operating ownership.