[Registered Patent] On-Device AI-Based Real-Time Manufacturing Defect Detection

Vision Transformer · Edge AI · Manufacturing Field Validation

This is an on-device AI technology that analyzes images captured at the manufacturing site internally within the device to determine the presence and location of defects. The goal was to reduce the latency and deployment burden of inspection methods relying on communication with high-performance servers, enabling on-site inspections even with limited computational resources.

Problem to Solve

Vision inspection in manufacturing sites requires both the analytical capability to identify minute defects and the processing performance to keep up with production speed. However, directly applying complex models to edge devices poses a significant burden in terms of computation volume and memory usage. Therefore, the core of this technology was to design a model architecture and processing method executable on edge hardware while utilizing the features necessary for defect analysis.

Core Technologies of the Patent

• Defect analysis utilizing multi-layer features
  It combines features extracted from multiple encoder layers of the Vision Transformer. It is designed to reflect the importance of features per layer through a Cross-Attention structure using the CLS token as a query.
• Computational architecture optimized for NPU execution
  Considering fixed-size token-based information transfer and parallel processing, a structure executable within limited computational resources was proposed.
• Calibration responding to environmental changes
  It includes a configuration that dynamically calibrates internal model parameters by reflecting environmental variables such as lighting, temperature, surface reflectance, and shooting distance.
• Visualization of defect location and condition
  It is configured to display suspected areas with a defect heatmap and evaluate the condition utilizing the area, intensity, and frequency of occurrence of the defects.

Product Implementation and Validation

In the related NUVION project, devices were installed in an operational factory to validate image collection and anomaly detection under conditions of vibration, cutting fluid, and irregular lighting. Based on presentation materials, an image-level anomaly detection rate of over 98% was confirmed on actual field data.

Furthermore, by applying TensorRT and FP16 on a Jetson Orin Nano, an inference speed of 29 FPS was reported, representing a 2.6x increase compared to the baseline. This project validated the technological applicability by connecting not only the detection performance of the model but also field installation and edge inference.