Multi-Modal Edge AI Based Substation Monitoring System Using Thermal-RGB Fusion for Intelligent Fault Detection and Predictive Maintenance

The Multi-Modal Edge AI Based Substation Monitoring System is an intelligent real-time monitoring and predictive maintenance solution designed for modern electrical substations and smart-grid infrastructure. The system combines thermal imaging, RGB visual inspection, and edge artificial intelligence to detect electrical faults, abnormal operating conditions, and equipment degradation with high accuracy and low latency.

The proposed architecture integrates an ESP32 microcontroller with an MLX90640 thermal infrared sensor to continuously monitor temperature variations and thermal hotspots from critical substation equipment such as transformers, insulators, circuit breakers, and busbars. Simultaneously, an RGB camera connected to a laptop captures live visual data for surface defect analysis and equipment inspection.

The laptop operates as an edge AI processing unit, where advanced computer vision and deep learning algorithms developed using OpenCV, TensorFlow Lite, and PyTorch analyze both thermal and RGB data streams in real time. By implementing multi-modal thermal-RGB data fusion, the system significantly improves fault detection accuracy by correlating thermal abnormalities with visible defects including insulation degradation, cracks, contamination, overheating, and loose electrical connections.

A real-time web-based monitoring dashboard developed using Flask, HTML, CSS, and JavaScript provides live visualization of thermal heatmaps, RGB camera streams, AI-based fault alerts, equipment monitoring status, and sensor analytics. The system also incorporates relay-based protection and buzzer alert mechanisms to provide immediate response during abnormal conditions.

The proposed solution offers low-latency edge processing, reduced cloud dependency, enhanced operational safety, improved reliability, and intelligent predictive maintenance capability. This project serves as a scalable and cost-effective framework for future smart-grid systems, Industry 4.0 applications, automated diagnostics, and next-generation intelligent electrical infrastructure monitoring.