SemiconductorsIndustrial vision →

Fully on-prem visual inspection of wafers

Rebuilding and stabilizing a vision pipeline on local GPU infrastructure. Unblocking a PoC abandoned for memory saturation: no cloud, no MES rework.

×2,3

inspection throughput on the same volume

−55 %

cycle time per image

GPU OOM errors after tuning

Context

The problem

Bottleneck at manual inspection

Operators worked through folders image by image via ad hoc scripts. No standard interface, no consistent model across teams.

Underused GPU, failing batches

The GPU hardware on the fab LAN sat largely idle: every batch-32 attempt ended in OOM and forced a fallback to single-image processing.

7+ minutes per batch of 108 images

Three wafers, 108 images: over seven minutes on a clean run, with retries whenever the GPU failed.

No compliance audit trail

No operator identifier, no terminal log, no per-image trace: quality reviews had nothing to work with.

Solution

Deployed architecture

Preparation

Parallel tile splitting

10-way split, CHW normalization on CPU (Parallel.For). Batch preparation without blocking the GPU thread.

Inference

Native C++ TritonClientWrapper

A DLL compiled against the Triton SDK (gRPC++, protobuf). Replaces HTTP REST calls from C#: reduced latency and marshalling. Batch-32 stable after tuning the 2 GB CUDA pool and ONNX BFCArena.

Model

YOLOv8 ONNX: 6 defect classes

Carrier scratches, vertical scratches, cracks, chips, particles, other marks. Confidence threshold 0.7, bounding boxes on full-resolution images.

Output

JPEG, CSV, audit summary

End-to-end processing on the fab LAN. Input folder → annotated output images. No cloud calls. Per-run log for quality review.

Results

Measured results

Measured in production conditions on the deployed scope.

62–66 s

Cycle per image

Versus 135–152 s before optimization: measured on the same reference batch.

~3,2 min

Batch time, 108 images

Versus 7+ minutes with retries and single-shot fallback.

Batch-32

Success on first attempt

After tuning the memory pool, single instance and ONNX BFCArena allocation.

Air-gapped

On-prem deployment

No MES rework, no developer tooling exposed to the floor. In production from the first deployment.

Lessons learned

What we took away

OOM failures often come from the call path (managed REST) and an ill-sized CUDA pool: not from the model itself.

A native gRPC wrapper to Triton pays off as soon as the batch exceeds a few images per second.

Operator/terminal traceability must be designed alongside the pipeline: not bolted on afterwards.

Do not abandon a GPU PoC without a memory audit (fragmentation, instance count, ONNX allocation strategy).

On-prem deployment demands a polished operator UX: folder in → folder out, with no command line.

Get started

A similar case on your site?

Tell us about the process, the data available and your deployment constraints. We will assess feasibility within 48 hours.